Storage device, data processing apparatus, data processing method, program, storage medium, and data processing system

ABSTRACT

A video processor card stores tap generation information for determining a predetermined tap coefficient when the tap generation information is used together with tap generation information stored in another video processor card, and supplies a video processing interface with the tap generation information. The video processing interface generates the tap coefficient from the tap generation information of the one video processor card and the other video processor card loaded therein. The video processing interface extracts video data having a predictive tap used to predict a target pixel and video data having a class tap used to classify the target data, and class classifies the target data based on the class tap. The video processing interface determine the target pixel based on the tap coefficient and the predictive tap of the class of the target pixel.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to a storage device, dataprocessing apparatus, data processing method, program, storage medium,and data processing system and, more particularly to a storage device,data processing apparatus, data processing method, program, storagemedium, and data processing system for allowing television receivers toperform sophisticated functions in an add-on fashion.

[0003] 2. Description of the Related Art

[0004] In response to television broadcast signals, television receiverspresent the image corresponding to a television broadcast signal andprovide a sound output correspondingly associated with the image.

[0005] A Y/C separation process and other signal processing rapidlyadvances, and sophisticated television receivers performing excellentsignal processing are developed one after another and are commerciallyavailable.

[0006] Even if a television receiver, more sophisticated than a modelcurrently owned by a user, is commercially available, the user is unableto enjoy a sophisticated function unless the user retrofits the presentmodel with a new one.

[0007] In the field of computers, a board having a predeterminedfunction, such as a video processor board or a audio processor board,may be mounted. A video processor board that receives a televisionsignal and performs an MPEG encoding operation may be installed in acomputer. That computer records a television broadcast program on a realtime basis while MPEG encoding the program.

[0008] As in the computer, it is contemplated that a board provided witha new function may be fitted into the television receiver. The userpurchases a board capable of performing a signal processing and mountsit in his or her television receiver. The user thus enjoys the newfunction without the need for purchasing a new model.

[0009] If a new board having more sophisticated function is on salesubsequent the sale of a preceding board having a certain signalprocessing function, the preceding board becomes obsolete.

[0010] The signal processing technique rapidly advances. One highlysophisticated board quickly supersedes another. The motivation of theusers to purchase the boards is thus reduced.

[0011] If a previously purchased board is usable with the functionthereof taken advantage of, or if the function of the televisionreceiver becomes sophisticated in function by adding a newly purchasedboard in addition to a previously purchased board, the motivation of theuser to purchase the board is promoted.

SUMMARY OF THE INVENTION

[0012] Accordingly, it is an object of the present invention to make atelevision receiver sophisticated in function in an add-on fashion.

[0013] In a first aspect of the present invention, a storage deviceincludes a tap generation information storage unit which stores tapgeneration information for generating a tap coefficient for eachpredetermined class for a data conversion process of converting firstdata into second data higher in quality level than the first data, a tapcoefficient generator which generates the tap coefficient from the tapgeneration information under the control of the data processingapparatus, a tap extractor which extracts, from the first data suppliedfrom the data processing apparatus, a predictive tap which is used topredict target data which is of interest in the second data, a class tapextractor which extracts, from the first data supplied from the dataprocessing apparatus, a class tap which is used to classify the targetdata into one of a plurality of classes, a class classifier whichclassifies the target data according to the class tap, and a predictorwhich predicts the target data from the tap coefficient and thepredictive tap of the class of the target data, and supplies the dataprocessing apparatus with the target data.

[0014] In a second aspect of the present invention, a data processingmethod includes a tap coefficient generation step of generating the tapcoefficient from the tap generation information under the control of thedata processing apparatus, a predictive tap extraction step ofextracting, from the first data supplied from the data processingapparatus, a predictive tap for use in the prediction of target datawhich is of interest in the second data, a class tap extraction step ofextracting, from the first data supplied from the data processingapparatus, a class tap for use in the class classification thatclassifies the target data into one of a plurality of classes, a classclassification step of classifying the target data into classes based onthe class tap, and a prediction step of predicting the target data fromthe tap coefficient and the predictive tap of the class of the targetdata, and supplying the target data to the data processing apparatus.

[0015] In a third aspect of the present invention, a computer programincludes a tap coefficient generation step of generating the tapcoefficient from the tap generation information under the control of thedata processing apparatus, a predictive tap extraction step ofextracting, from the first data supplied from the data processingapparatus, a predictive tap for use in the prediction of target datawhich is of interest in the second data, a class tap extraction step ofextracting, from the first data supplied from the data processingapparatus, a class tap for use in the class classification thatclassifies the target data into one of a plurality of classes, a classclassification step of classifying the target data into classes based onthe class tap, and a prediction step of predicting the target data fromthe tap coefficient and the predictive tap of the class of the targetdata, and supplying the target data to the data processing apparatus.

[0016] In a fourth aspect of the present invention, a storage mediumstores a computer program of a data processing method, and includes atap coefficient generation step of generating the tap coefficient fromthe tap generation information under the control of the data processingapparatus, a predictive tap extraction step of extracting, from thefirst data supplied from the data processing apparatus, a predictive tapfor use in the prediction of target data which is of interest in thesecond data, a class tap extraction step of extracting, from the firstdata supplied from the data processing apparatus, a class tap for use inthe class classification that classifies the target data into one of aplurality of classes, a class classification step of classifying thetarget data into classes based on the class tap, and a prediction stepof predicting the target data from the tap coefficient and thepredictive tap of the class of the target data, and supplying the targetdata to the data processing apparatus.

[0017] In a fifth aspect of the present invention, a data processingapparatus includes a loading and unloading unit on which each of thefirst through N-th storage devices is mounted, a tap coefficientgeneration control unit which controls the generation of the tapcoefficient from the tap generation information in the first throughN′-th storage devices (N′≦N) mounted on the loading and unloading unit,an input and output route setting unit which sets an input and outputroute of data for each of the first through N′-th storage devices, and adata supply control unit which controls the supply of data from onestorage device to another among the first through N′-th storage devicesin accordance with the input and output route set by the input andoutput route setting unit.

[0018] In a sixth aspect of the present invention, a data processingmethod includes a tap coefficient generation control step of controllingthe generation of the tap coefficient from the tap generationinformation in the first through N′-th storage devices (N′≦N) mounted onthe loading and unloading unit, an input and output route setting stepof setting an input and output route of data for each of the firstthrough N′-th storage devices, and a data supply control step ofcontrolling the supply of data from one storage device to another amongthe first through N′-th storage devices in accordance with the input andoutput route set in the input and output route setting step.

[0019] In a seventh aspect of the present invention, a computer programfor a data processing method includes a tap coefficient generationcontrol step of controlling the generation of the tap coefficient fromthe tap generation information in the first through N′-th storagedevices (N′≦N) mounted on the loading and unloading unit, an input andoutput route setting step of setting an input and output route of datafor each of the first through N′-th storage devices, and a data supplycontrol step of controlling the supply of data from one storage deviceto another among the first through N′-th storage devices in accordancewith the input and output route set in the input and output routesetting step.

[0020] In an eight aspect of the present invention, a storage mediumstores a computer program for data processing method and includes a tapcoefficient generation control step of controlling the generation of thetap coefficient from the tap generation information in the first throughN′-th storage devices (N′≦N) mounted on the loading and unloading unit,an input and output route setting step of setting an input and outputroute of data for each of the first through N′-th storage devices, and adata supply control step of controlling the supply of data from onestorage device to another among the first through N′-th storage devicesin accordance with the input and output route set in the input andoutput route setting step.

[0021] In a ninth aspect of the present invention, a data processingsystem includes first through N-th storage devices which store tapgeneration information for generating a tap coefficient for eachpredetermined class for a data conversion process of converting firstdata into second data higher in quality level than the first data, and adata processing apparatus that allows the first through N-th storagedevices to be detachably loaded therein. Each of the first through N-thstorage device includes tap generation information storage unit whichstores the tap generation information for generating the tapcoefficient, a tap coefficient generator which generates the tapcoefficient from the tap generation information under the control of thedata processing apparatus, a tap extractor which extracts, from thefirst data supplied from the data processing apparatus, a predictive tapwhich is used to predict target data which is of interest in the seconddata, a class tap extractor which extracts, from the first data suppliedfrom the data processing apparatus, a class tap which is used toclassify the target data into one of a plurality of classes, a classclassifier which classifies the target data according to the class tap,and a predictor which predicts the target data from the tap coefficientand the predictive tap of the class of the target data, and supplies thedata processing apparatus with the target data. The data processingapparatus includes a loading and unloading unit on which each of thefirst through N-th storage devices is mounted, a tap coefficientgeneration control unit which controls the generation of the tapcoefficient from the tap generation information in the first throughN′-th storage devices (N′≦N) mounted on the loading and unloading unit,an input and output route setting unit which sets an input and outputroute of data for each of the first through N′-th storage devices, and adata supply control unit which controls the supply of data from onestorage device to another among the first through N′-th storage devicesin accordance with the input and output route set by the input andoutput route setting unit.

[0022] In a tenth aspect of the present invention, a storage deviceincludes a tap generation information storage unit which stores tapgeneration information for generating a tap coefficient for eachpredetermined class for a data conversion process of converting firstdata into second data higher in quality level than the first data, thetap coefficient being generated from the tap generation informationstored in the storage device and the tap generation information storedin another storage device, and a tap generation information supply unitwhich supplies the data processing apparatus with the tap generationinformation.

[0023] In an eleventh aspect of the present invention, a data processingapparatus includes a loading and unloading unit on which each of thefirst through N-th storage devices is mounted, a tap coefficientgenerator unit which generates the tap coefficient from the tapgeneration information in the first through N′-th storage devices (N′≦N)mounted on the loading and unloading unit, a predictive tap extractorwhich extracts the first data having a predictive tap which is used topredict target data which is of interest in the second data, a class tapextractor which extracts the first data having a class tap which is usedto classify the target data into one of a plurality of classes, a classclassifier which classifies the target data based on the class tap, anda predictor which predicts the target data based on the tap coefficientand the predictive tap of the class of the target data.

[0024] In a twelfth aspect of the present invention, a data processingmethod includes a tap coefficient generation step of generating the tapcoefficient from the tap generation information in the first throughN′-th storage devices (N′≦N) loaded in the data processing apparatus, apredictive tap extracting step of extracting the first data having apredictive tap which is used to predict target data which is of interestin the second data, a class tap extracting step of extracting the firstdata having a class tap which is used to classify the target data intoone of a plurality of classes, a class classifying step of classifyingthe target data based on the class tap, and a predicting step ofpredicting the target data based on the tap coefficient and thepredictive tap of the class of the target data.

[0025] In a thirteenth aspect of the present invention, a computerprogram of a data processing method includes a tap coefficientgeneration step of generating the tap coefficient from the tapgeneration information in the first through N′-th storage devices (N′≦N)loaded in the data processing apparatus, a predictive tap extractingstep of extracting the first data having a predictive tap which is usedto predict target data which is of interest in the second data, a classtap extracting step of extracting the first data having a class tapwhich is used to classify the target data into one of a plurality ofclasses, a class classifying step of classifying the target data basedon the class tap, and a predicting step of predicting the target databased on the tap coefficient of and the predictive tap of the class ofthe target data.

[0026] In a fourteenth aspect of the present invention, a storage mediumstores a computer program for a data processing method and includes atap coefficient generation step of generating the tap coefficient fromthe tap generation information in the first through N′-th storagedevices (N′≦N) loaded in the data processing apparatus, a predictive tapextracting step of extracting the first data having a predictive tapwhich is used to predict target data which is of interest in the seconddata, a class tap extracting step of extracting the first data having aclass tap which is used to classify the target data into one of aplurality of classes, a class classifying step of classifying the targetdata based on the class tap, and a predicting step of predicting thetarget data based on the tap coefficient and the predictive tap of theclass of the target data.

[0027] In a fifteenth aspect of the present invention, a data processingsystem includes first through N-th storage devices storing tapgeneration information for generating a tap coefficient for eachpredetermined class for a data conversion process of converting firstdata into second data higher in quality level than the first data, and adata processing apparatus on which the first through N-th storagedevices are detachably loaded. Each of the first through N-th storagedevices includes a tap generation information storage unit which storestap generation information for generating the tap coefficient, the tapcoefficient being generated from the tap generation information and tapgeneration information stored in another storage device, and a tapgeneration information supply unit which supplies the data processingapparatus with the tap generation information. The data processingapparatus includes a loading and unloading unit on which each of thefirst through N-th storage devices is mounted, a tap coefficientgenerator which generates the tap coefficient from the tap generationinformation in the first through N′-th storage devices (N′≦N) mounted onthe loading and unloading unit, a predictive tap extractor whichextracts the first data having a predictive tap which is used to predicttarget data which is of interest in the second data, a class tapextractor which extracts the first data having a class tap which is usedto classify the target data into one of a plurality of classes, a classclassifier which classifies the target data based on the class tap, anda predictor which predicts the target data based on the tap coefficientand the predictive tap of the class of the target data.

[0028] In accordance with the first through fourth aspects of thepresent invention, the tap coefficient is produced from the tapgeneration information under the control of the data processingapparatus. The predictive tap used to predict the target data which isof interest in the second data and the class tap used to classify thetarget data into one of the plurality of classes are extracted from thefirst data supplied from the data processing apparatus. Based on theclass tap, the target data is classified, and based on the tapcoefficient and the predictive tap of the class of the target data, thetarget data is predicted and then fed to the data processing apparatus.

[0029] In accordance with the first, sixth, seventh, and eighth aspectsof the present invention, the generation of the tap coefficient from thetap generation information in the first through N′-th storage devices(N′≦N) mounted on the loading and unloading unit is controlled, and theinput and output route of data for each of the first through N′-thstorage devices is set. The supply of data from one storage device toanother among the first through N′-th storage devices is controlled inaccordance with the input and output route set by the input and outputroute setting unit.

[0030] In accordance with the data processing system of the ninth aspectof the present invention, the tap coefficient is generated from the tapgeneration information in each of the first through N-th storage devicesunder the control of the data processing apparatus. The predictive tapused to predict the target data, which is of interest in the seconddata, and the class tap used to classify the target data into one of theplurality of classes are extracted from the first data supplied from thedata processing apparatus. Based on the class tap, the target data isclassified. The target data is predicted from the tap coefficient andthe predictive tap of the class of the target data, and is then fed tothe data processing apparatus. In the data processing apparatus, thegeneration of the tap coefficient from the tap generation information iscontrolled in the first through N′-th storage devices (N′≦N) mountedthereon. The input and output route of data is respectively set for eachof the first through N′-th storage devices. Using the set input andoutput routes, the supply of data from the one storage device to anotherstorage device is controlled among the first through N′-th storagedevices.

[0031] In accordance with the storage device of the tenth aspect, thetap generation information storage unit stores the tap generationinformation for generating the tap coefficient for each predeterminedclass for the data conversion process of converting the first data intothe second data higher in quality level than the first data. The tapcoefficient is generated from the tap generation information in thestorage device and tap generation information in another storage device.The tap generation information is then fed to the data processingapparatus.

[0032] In accordance with the eleventh through fourteenth aspects of thepresent invention, the tap coefficient is generated from the tapgeneration information in the first through N′-th storage devices (N′≦N)mounted on the loading and unloading unit. The first data having thepredictive tap which is used to predict the target data which is ofinterest in the second data is extracted. The first data having a classtap which is used to classify the target data into one of a plurality ofclasses is extracted. The target data is classified based on the classtap. The target data is predicted based on the tap coefficient and thepredictive tap of the class of the target data.

[0033] In accordance with the data processing system of the fifteenthaspect of the present invention, each of the first through N-th storagedevices stores the tap generation information for generating the tapcoefficient. The tap coefficient is generated from the tap generationinformation in the storage device and tap generation information storedin another storage device. The data processing apparatus is suppliedwith the tap generation information. The tap coefficient is generatedfrom the tap generation information in the first through N′-th storagedevices (N′≦N) mounted on the loading and unloading unit. The first datahaving the predictive tap which is used to predict the target data whichis of interest in the second data is extracted. The first data having aclass tap which is used to classify the target data into one of aplurality of classes is extracted. The target data is classified basedon the class tap. The target data is predicted based on the tapcoefficient and the predictive tap of the class of the target data.

BRIEF DESCRIPTION OF THE DRAWINGS

[0034]FIG. 1 is a perspective view illustrating the construction of atelevision receiver in accordance with one embodiment of the presentinvention;

[0035]FIG. 2 is a rear view of a main unit of the television receiver;

[0036]FIG. 3 is an electrical block diagram of the main unit;

[0037]FIG. 4 is a plan view of a remote controller;

[0038]FIG. 5 is a block diagram illustrating a first construction of avideo processing interface 40;

[0039]FIG. 6 is a block diagram illustrating a first construction of avideo processor card 13;

[0040]FIG. 7 is a flow diagram illustrating the process of a videointerface 40;

[0041]FIG. 8 is a flow diagram illustrating the process of the videoprocessor card 13;

[0042]FIG. 9 is a block diagram illustrating a learning device fordetermining a tap coefficient;

[0043]FIG. 10 is a flow diagram illustrating a learning process fordetermining a tap coefficient;

[0044]FIG. 11 is a block diagram illustrating the construction of thelearning device for determining coefficient seed data;

[0045]FIG. 12 is a flow diagram illustrating a learning process fordetermining the coefficient seed data;

[0046]FIG. 13 illustrates tap coefficients generated from learning dataand training data, and the coefficient seed data;

[0047]FIG. 14 illustrates a generation method of the coefficient seeddata stored in the video processor card 13;

[0048]FIG. 15 illustrates the generation method of the coefficient seeddata stored in the video processor card 13;

[0049]FIG. 16 illustrates the generation method of the coefficient seeddata stored in the video processor card 13;

[0050]FIG. 17 illustrates the process of a video processor card 13 ₁;

[0051]FIG. 18 illustrates the process of video processor cards 13 ₁ and13 ₂;

[0052]FIG. 19 illustrates the generation method for generating thecoefficient seed data from difference data;

[0053]FIG. 20 is a block diagram illustrating a second construction ofthe video processor card 13;

[0054]FIG. 21 illustrates the process of the video processor card 13 ₁;

[0055]FIG. 22 illustrates the process of the video processor cards 13 ₁and 13 ₂;

[0056]FIG. 23 illustrates real memory space of the video processor card13 ₁;

[0057]FIG. 24 illustrates the real memory space of the video processorcard 13 ₁;

[0058]FIG. 25 illustrates real memory space of the video processor card13 ₂;

[0059]FIG. 26 illustrates the video processor cards 13 ₁ and 13 ₂ whichstore a tap coefficient in the virtual memory space thereof;

[0060]FIG. 27 is a flow diagram illustrating the process of the videoprocessor card 13;

[0061]FIG. 28 illustrates the generation method of the coefficient seeddata stored in the video processor card 13;

[0062]FIG. 29 illustrates the generation method of the coefficient seeddata stored in the video processor card 13;

[0063]FIG. 30 illustrates the generation method of the coefficient seeddata stored in the video processor card 13;

[0064]FIG. 31 illustrates the generation method of the coefficient seeddata stored in the video processor card 13;

[0065]FIG. 32 is a block diagram illustrating a second construction ofthe video processing interface 40;

[0066]FIG. 33 is a block diagram illustrating a third construction ofthe video processor card 13;

[0067]FIG. 34 is a flow diagram illustrating the process of the videoprocessing interface 40;

[0068]FIG. 35 is a flow diagram illustrating the process of the videoprocessor card 13;

[0069]FIG. 36 is a chart explaining that the number of classes increaseswith the number of mounted video processor cards 13;

[0070]FIG. 37 illustrates the process of the video processor card 13 ₁and the video processing interface 40;

[0071]FIG. 38 illustrates the process of the video processor card 13 ₁and 13 ₂, and the video processing interface 40;

[0072]FIG. 39 illustrates the process of the video processor card 13 ₁and 13 ₂, and the video processing interface 40;

[0073]FIG. 40 illustrates the relationship between the number of taps ofa first synthesis class code and a second synthesis class code;

[0074]FIG. 41 is a flow diagram illustrating the method of generatingtap generation information;

[0075]FIG. 42 is a flow diagram illustrating the process of the videoprocessing interface 40 and coefficient generator 136;

[0076]FIG. 43 is a block diagram illustrating a third construction ofthe video processing interface 40;

[0077]FIG. 44 is a block diagram illustrating a fourth construction ofthe video processor card 13; and

[0078]FIG. 45 is a block diagram illustrating the construction of oneembodiment of a computer implementing the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0079]FIG. 1 is the external view of a television receiver of oneembodiment of the present invention.

[0080] As shown, the television receiver includes a main unit 1 and aremote controller 2.

[0081] A CRT (Cathode Ray Tube) 11 is mounted on the front of the mainunit 1, and a video such as a television program is presented on the CRT11.

[0082] Six slots 12 ₁ through 12 ₆ are arranged on the bottom of themain unit 1. A video processor card 13 _(i) is loaded into and unloadedfrom the slot 12 _(i).

[0083] In the embodiment shown in FIG. 1, a video processor card 13 ₁ isalready mounted in the slot 12 ₁ of the six slots 12 ₁ through 12 ₆ in adefault setting. The slot 12 ₁ is closed with a cover so that the videoprocessing card 13 ₁ is not easily pulled out.

[0084] Although the six slots 12 ₁ through 12 ₆ are arranged in theembodiment shown in FIG. 1, the number of slots is not limited to six.Five slots or less or seven slots or more may be used.

[0085] The video processing card 13 is an IC (Integrated Circuit) cardor a memory card, which adds functions to the main unit 1 of thetelevision receiver. With the video processing card 13 inserted into theslot 12 _(i), the user enjoys a diversity of functions as will bediscussed later.

[0086] The video processing card 13 complies with existing standardssuch as PCMCIA (Personal Computer Memory Card International Association)standards. Alternatively, the video processing card 13 may be the onecomplying with any other standards such as in-house standards.

[0087] The remote controller 2 is operated to change a receiving channelor volume control setting, or to input other commands to the main unit1. In response to an operational input, the remote controller 2 emitsinfrared light. The main unit 1 receives the infrared light, and carriesout a process responsive to the operation of the remote controller 2.

[0088] The remote controller 2 may use radiowave in compliance withBluetooth (Trade Name), instead of infrared light.

[0089]FIG. 2 is a rear view of the main unit 1 of FIG. 1.

[0090] Arranged on the rear side of the main unit 1 are an antennaterminal 21 to which an antenna (not shown) is connected through acable, input terminals 22 for receiving video and audio signals in themain unit 1, and output terminals 23 for outputting video and audiosignals from the main unit 1.

[0091] The slots 12 ₁ through 12 ₆, which are arranged on the frontpanel of the main unit 1 as shown in the embodiment shown in FIG. 1, maybe arranged on the rear panel of the main unit 1 as shown in FIG. 2.

[0092] Rather than installing all slots 12 ₁ through 12 ₆ on either thefront panel or the rear panel, some of the slots may be arranged on thefront panel, and the remaining slots may be arranged on the rear panel.

[0093] The video processor card 13 _(i) may be constructed to be like aPCI (Peripheral Component Interconnect) card, which is typically used asan expanded memory for computers. In this case, the slot 12 _(i) must beconstructed to receive a PCI card.

[0094]FIG. 3 is an electrical block diagram of the main unit 1 of FIG.1.

[0095] A tuner 31 receives a signal from the antenna (not shown) throughthe antenna terminal 21. Under the control of a controller 37, the tuner31 detects and decodes a television broadcast signal at a predeterminedchannel, and supplies an A/D (analog-to-digital) converter 32 with adecoded signal.

[0096] The A/D converter 32 analog-to-digital converts the televisionbroadcast signal from the tuner 31, and supplies a Y/C separator 33 withvideo data of the resulting digital signal.

[0097] The A/D converter 32 also outputs audio data of the televisionbroadcast signal to a loudspeaker (not shown).

[0098] The Y/C separator 33 Y/C performs Y/C separation process on thevideo data from the A/D converter 32 and feeds the Y/C separated signalsto a selector 34. Under the control of the controller 37, the selector34 selects between the video data supplied from the Y/C separator 33 andthe video data supplied through the input terminals 22, and then feedsthe selected video data to a frame memory 35.

[0099] The frame memory 35 stores the video data fed from the selector34, and feeds the video data to the output terminals 23 and matrixconverter 36. The frame memory 35 stores the video data supplied fromthe selector 34, and then feeds the stored video data to a videoprocessing interface 40. Further, the frame memory 35 stores video datafrom the video processing interface 40 after being subjected topredetermined video processing, and then feeds that video data to theoutput terminals 23 and the matrix converter 36.

[0100] The frame memory 35 contains at least first through fourth banksfor storing one frame (field) of video data. The frame memory 35 storesthe video data from the selector 34 alternately on the first bank andthe second bank. The video data is read from one of the first and secondbanks which has no video data written from the selector 34, and is fedto the video processing interface 40. The frame memory 35 stores thevideo data output from the video processing interface 40 alternately onthe third and fourth banks. The video data is read from one of the thirdand fourth banks which has no video data written from the videoprocessing interface 40, and is fed to the matrix converter 36.

[0101] The frame memory 35 performs bank switching in this way, therebywriting thereto the video data supplied from the selector 34, readingthe video data therefrom to the video processing interface 40, writingthe video data thereto from the video processing interface 40, andreading the video data therefrom to the matrix converter 36 on a realtime basis.

[0102] The matrix converter 36 converts the video data supplied from theframe memory 35 into RGB video data, and then converts the RGB videodata into analog data. The video data output from the matrix converter36 is fed to the CRT 11.

[0103] The controller 37 includes a CPU (Central Processing Unit) 37A,EEPROM (Electrically Erasable Programmable Read Only Memory) 37B, andRAM (Random Access Memory) 37C, and controls the tuner 31, selector 34,communication interface 38, and video processing interface 40.

[0104] Various processes are performed under the control of programsstored in the CPU 37A and EEPROM 37B. The tuner 31, selector 34,communication interface 38, and video processing interface 40 are thuscontrolled. The CPU 37A performs a process responsive to a command fedfrom an IR (Infrared Ray) interface 39. The CPU 37A controls thecommunication interface 38, thereby accessing a server (not shown)through a telephone line, for example, and acquiring an upgraded programand required data from the server.

[0105] The EEPROM 37B stores programs and data which must be stored evenafter power off. Program upgrading may be performed by simplyoverwriting existing ones by new programs and data.

[0106] The RAM 37C temporarily stores the data and program required bythe CPU 37A in operation.

[0107] For example, the communication interface 38 includes an analogmodem, ADSL (Asymmetrical Digital Subscriber Line) modem, DSU (DigitalService Unit), TA (Terminal Adapter), LAN (Local Area Network), etc.Under the control of the controller 37, the communication interface 38controls communication through a telephone line or other communicationline.

[0108] In response to infrared light from the remote controller 2, theIR interface 39 photoelectrically converts the infrared light to anelectrical signal and feeds the electrical signal to the controller 37.

[0109] The video processing interface 40 has slots 12 ₁ through 12 ₆ inwhich video processor cards 13 ₁ through 13 ₆ are respectively inserted.Together with a video processor card 13 _(i) inserted into a slot 12_(i), the video processing interface 40 performs video processing (dataconversion process) on the video data stored in the frame memory 35 aswill be discussed later.

[0110] The video processor card 13 _(i) is inserted into the slot 12_(i) in the embodiment shown in FIG. 3. The video processor card 13 _(i)may be inserted into any of the six slots 12 ₁ through 12 ₆. In thisembodiment, the video processor card 13 _(i) is inserted into the slot12 _(i) for simplicity of explanation.

[0111] In this embodiment, the video processing interface 40 had the sixslots 12 ₁ through 12 ₆ into which the six types of video processorcards 13 ₁ through 13 ₆ may be respectively inserted. In the embodimentshown in FIG. 3, three types of video processor cards 13 ₁ through 13 ₃are respectively inserted into the slot 12 ₁ through 12 ₃.

[0112] The slot 12 _(i) has predetermined terminals therewithin in thisembodiment. With the video processor card 13 _(i) put into physicalcontact with these terminals, the video processor card 13 _(i) iselectrically connected to the video processing interface 40 to exchangea variety of pieces of data therebetween. Alternatively, the exchange ofdata between the video processor card 13 _(i) and the video processinginterface 40 may be performed using radio communication.

[0113]FIG. 4 is a plan view of the remote controller 2.

[0114] A selection button switch 51 operates in eight directions,namely, vertically up and down, horizontally leftward and rightward, andfour diagonal directions (directional operations). The selection buttonswitch 51 can be vertically pressed with respect to the top surface ofthe remote controller 2 (selection operation). A menu button switch 54is operated to present, on the CRT 11 of the main unit 1, a menu screenon which a variety of settings and commands for performing apredetermined process are input.

[0115] When the menu screen is presented, a cursor appears on the CRT 11to point to an item of a menu. When the selection button switch 51 isoperated for the directional operation, the cursor moves in a directioncorresponding to the directional operation. If the selection buttonswitch 51 is operated in the selection operation with the cursor placedon a predetermined item, the selection of the corresponding item isinput. Icons may be presented on the menu. When the selection buttonswitch 51 clicks an icon, the selection operation is carried out.

[0116] An exit button switch 55 is operated to return to a standardscreen from the menu screen.

[0117] A volume button switch 52 is operated for high volume setting orfor low volume setting. A channel up/down button switch 53 is operatedto change the receiving channel for high channel number or low channelnumber.

[0118] Numeric button switches 58 labeled 0 through 9 are operated toinput corresponding numbers. The operation of each numeric button switch58 is followed by the pressing of an ENTER button switch 57 to enter thecorresponding number really. When the channel is switched, a new channelnumber is presented in OSD (On Screen Display) on the screen of the CRT11 of the main unit 1 for a predetermined duration of time. A displaybutton 56 switches on and off the OSD of the currently selected channelnumber and volume level.

[0119] A TV/video switching button 59 is used to switch the input to themain unit 1 between the signal from the tuner 31 and the signal from theinput terminals 22. A TV/DSS switching button switch 60 is operated toswitch between a TV mode for receiving ground waves through the tuner 31and DSS mode (Digital Satellite System (Trade Name of HughesCommunications)) for receiving satellite broadcasting. When channelswitching is performed operating the numeric button switched 58, thechannel prior to channel switching is stored. A jump button switch 61 isused to return to the prior channel.

[0120] A language button 62 is used to select a predetermined languagewhen broadcasting is performed in two or more languages. A guide buttonswitch 63 is operated to display an EPG (Electronic Program Guide) onthe CRT 11. A favorite button switch 64 is operated to select apredetermined channel preferred by the user.

[0121] A cable button switch 65, TV switch 66, and DSS button switch 67are used to switch the apparatus category indicated by a command codecorresponding to infrared light emitted from the remote controller 2.Specifically, the remote controller 2 is able to remote control an STBor IRD (not shown) besides the main unit 1 of the television receiver.The cable button switch 65 is used to control a STB (Set Top Box) thatreceives a signal coming in through a CATV network with the remotecontroller 2. Subsequent to the operation of the cable button switch 65,the remote controller 2 transmits infrared light corresponding to thecommand code of the apparatus category assigned to the STB. Similarly,the TV switch 66 is used when the main unit 1 is controlled by theremote controller 2. The DSS button switch 67 is used when the remotecontroller 2 controls the IRD (Integrated Receiver and Recorder) thatreceives a signal transmitted through a satellite.

[0122] LEDs (Light Emitting Diodes) 68, 69, and 70 are respectively litwhen the cable button switch 65, TV switch 66, and DSS button switch 67are turned on. These LEDs thus indicate to the user which apparatuscategory is currently controlled by the remote controller 2. The LEDS68, 69, and 70 are extinguished respectively when the cable buttonswitch 65, TV switch 66, and DSS button switch 67 are turned off.

[0123] A cable power-supply button switch 71, TV power-supply buttonswitch 72, and DSS power-supply button switch 73 are respectivelyoperated to switch on or off the power supplies of the STB, the mainunit 1, and the IRD.

[0124] A muting button switch 74 is operated to set or reset the mutingstate of the main unit 1. A sleep button switch 75 sets or resets asleep mode which automatically turns off the power when it becomes apredetermined time or a predetermined duration of time has elapsed.

[0125] A light emitter 76 emits infrared light corresponding to an inputon the remote controller 2 when the user operates the remote controller2.

[0126]FIG. 5 illustrates the construction of a first embodiment of thevideo processing interface 40 shown in FIG. 3.

[0127] An interface controller 81 controls a memory interface 82. Inresponse to an output from a connection detector 84, the interfacecontroller 81 detects the loading or unloading of the video processorcard 13 _(i) into or from the slot 12 _(i). In response to the detectionresult, the interface controller 81 controls a card interface 83. Theinterface controller 81 controls the exchange of data between the memoryinterface 82 and the card interface 83.

[0128] Under the control of the interface controller 81, the memoryinterface 82 reads video data from the frame memory 35 (see FIG. 3), andfeeds the video data to the interface controller 81. The memoryinterface 82 receives the video data supplied from the interfacecontroller 81, and feeds the video data to a line-by-line converter 85.

[0129] The card interface 83, controlled by the interface controller 81,supplies the video processor card 13 _(i) inserted in the slot 12 _(i),with the video data supplied from the interface controller 81.

[0130] The card interface 83, connected to the slots 12 ₁ through 12 ₆,receives the video data and a control signal supplied from the interfacecontroller 81, and feeds the video data and the control signal to thevideo processor card 13 _(i) inserted in the slot 12 _(i).

[0131] The card interface 83 receives data such as video data andcontrol signals from the video processor card 13 _(i) inserted in theslot 12 _(i), and then feeds the video data and control signals to theinterface controller 81 and another video processor card 13 _(j)inserted in another slot 12 _(j).

[0132] The card interface 83 supplies the connection detector 84 with avoltage (terminal voltage) at a terminal of the slot 12 _(i).

[0133] The connection detector 84 monitors the terminal voltage of eachof the slots 12 ₁ through 12 ₆ through the card interface 83, and checksto see if the video processor card 13 _(i) is loaded into or unloadedfrom the slot 12 _(i) in response to a change in the terminal voltage.The connection detector 84 feeds the detected result to the cardinterface 83. Alternatively, the detection of the loading and unloadingof the video processor card 13 _(i) may be mechanically performed.

[0134] The line-by-line converter 85 converts the scanning method of thevideo data supplied from the memory interface 82, for example, from aninterlace scanning method to a line-by-line scanning method(non-interlace scanning method), or from the line-by-line scanningmethod to the interlace scanning method, as appropriate, and feeds thevideo data to the frame memory 35 for storage there.

[0135]FIG. 6 illustrates the construction of the video processor card 13_(i) compatible with the video processing interface 40 constructed asshown in FIG. 5.

[0136] In the embodiment illustrated in FIG. 6, the video processor card13 _(i) performs a data conversion process on the video data suppliedthereto as first video data into second video data higher in qualitylevel than the first video data.

[0137] Let the first video data be a low-resolution video data and thesecond video data be a high-resolution video data, and the dataconversion process may become a resolution enhancement process. If thefirst video data has a low S/N ratio (signal to noise ratio) with thesecond video data having a high S/N ratio, the data conversion processmay become a noise removal process. If the first video data is videodata having a predetermined size with the second video data having asize smaller than or larger than the size of the first video data, thedata conversion process may become a resizing process of the video (forscale contraction or scale expansion).

[0138] The data conversion process serves various functions depending onthe definition of the first and second video data.

[0139] Tap extractors 91 and 92 receive the first video data, which mustbe subjected to the data conversion process, from the card interface 83(see FIG. 5).

[0140] The tap extractor 91 successively sets each pixel forming thesecond video data as the target pixel. The tap extractor 91 extracts, asa predictive tap, several pixels (values) forming the first pixel datawhich is used to predict the pixel value of the target pixel.

[0141] Specifically, the tap extractor 91 extracts, as the predictivetap, a plurality of pixels close in space or in time to a pixel of thefirst data corresponding to the target pixel (namely, a pixel of thefirst video data closest to a pixel at the same position as the targetpixel). For example, the plurality of pixels includes the pixelcorresponding to the target pixel, and pixels adjacent to that pixel.

[0142] The tap extractor 92 extracts, as a class tap, a plurality ofpixels forming the first data which is used to classify the targetpixels into a plurality of classes.

[0143] The card controller 98 feeds a control signal to the tapextractors 91 and 92. The tap structure of the predictive tapconstructed in the tap extractor 91 and the tap structure of the classtap constructed in the tap extractor 92 are set by the control signalfrom the card controller 98.

[0144] The predictive tap and the class tap may have the same tapstructure or different tap structures.

[0145] The predictive tap obtained by the tap extractor 91 is fed to apredictor 95, and the class tap obtained by the tap extractor 92 is fedto a class classifier 93.

[0146] The class classifier 93 classifies the target pixel based on theclass tap from the tap extractor 92, and feeds a resulting class codecorresponding to the class to a coefficient memory 94.

[0147] The class classification method may be ADRC (Adaptive DynamicRange Coding), for example.

[0148] In the method of using the ADRC, the pixel value of the pixelconstituting the class tap is ADRC processed, and the class of thetarget pixel is determined based on the resulting ADRC code.

[0149] In K bit ADRC processing, the maximum value MAX and the minimumvalue MIN of the pixel values of the pixels forming the class tap aredetected. DR=MAX−MIN is a localized dynamic range of a set, and thepixel value forming the class tap is re-quantized to K bits based on thedynamic range DR. Specifically, the minimum value MIN is subtracted fromthe pixel value of each pixel forming the class tap, and the remaindervalue is divided (quantized) by DR/2^(k). The pixel values of the pixelsof K bits forming the class tap are arranged in a bit train in apredetermined order, and are output as an ADRC code. For example, if aclass tap is processed using 1-bit ADRC processing, the minimum valueMIN is subtracted from the pixel value of the pixel forming that classtap and the remainder value is divided by the average of the maximumvalue MAX and the minimum value MIN (with decimal numbers rounded down)In this way, the pixel value of each pixel becomes 1 bit (binarized). Abit train in which 1-bit pixel values are arranged in the predeterminedorder is output as the ADRC code.

[0150] The class classifier 93 may output, as a class code, a pattern oflevel distribution of the pixel value of the pixel forming the classtap. If it is assumed that the class tap includes the pixel values of Npixels, and that K bits are allowed for the pixel value of the pixel,the number of class codes output from the class classifier 93 becomes(2^(N))^(K). The number of class codes becomes a large number whichexponentially increases with bit number K of the pixel value of thepixel.

[0151] The class classifier 93 preferably compresses the amount ofinformation of the class tap using the above-referenced ADRC processing,or vector quantization, for class classification.

[0152] As discussed above, the class classification is performed basedon the pattern of level distribution of the pixel values of the pixelsforming the class tap. Alternatively, the class classification may beperformed based on the presence or absence of an edge on the pixelcorresponding to the target pixel of the class tap, or based on whetherthe pixel moves (in terms of distance and direction).

[0153] The coefficient memory 94 stores the tap coefficient for eachclass supplied from a coefficient generator 96 through a coefficientnormalizer 99. The coefficient memory 94 supplies the predictor 95 withthe tap coefficient stored in an address corresponding to the class codesupplied from the class classifier 93 (namely, the tap coefficient ofthe class expressed by the class code supplied from the class classifier93), out of the tap coefficients stored therein. The tap coefficientcorresponds to a coefficient which is multiplied by input data in a tapin a digital filter.

[0154] The predictor 95 acquires the predictive tap output from the tapextractor 91 and the tap coefficient output from the coefficient memory94, and performs predetermined prediction calculation to determine apredictive value of the true value of the target pixel using thepredictive tap and the tap coefficient. In this way, the predictor 95determines and outputs (the predictive value of) the pixel value of thetarget pixel, namely, the pixel value of the pixel constituting thesecond video data.

[0155] The coefficient generator 96 generates the tap coefficient foreach class based on the tap generation information stored in a tapgeneration storage unit 97 and a parameter supplied from the cardcontroller 98, and then feeds the tap coefficient to the coefficientmemory 94 through the coefficient normalizer 99 to be written thereon inan overwrite fashion.

[0156] The tap generation storage unit 97 stores the tap generationinformation to generate the tap coefficient for each class.

[0157] The tap generation information may be coefficient seed data whichis a seed of the tap coefficient and determined through a learningprocess as will be discussed later, or may be information that willgenerate that coefficient seed data.

[0158] The slots 12 ₁ through 12 ₆ in the video processing interface 40and the video processor cards 13 ₁ through 13 ₆ are constructed asalready discussed with reference to FIG. 6. The tap generationinformation stored in the tap generation storage unit 97 generates tapcoefficients different from video processor card 13 to video processorcard 13.

[0159] The tap generation storage unit 97 stores the process informationwhich represents the content of the data conversion process which iscarried out by the video processor card 13 _(i) in video processing.

[0160] The process information contains the predictive tap generated(constructed) by the tap extractor 91, information representing the tapstructure of the class tap generated by the tap extractor 92,information representing the class classification method carried out bythe class classifier 93 (for example, information that classclassification is carried out based on the pattern of the leveldistribution of pixel values, the presence or absence of edges, orwhether or not the pixel moves), the number of classes of the tapcoefficients generated from the tap generation information, etc.

[0161] The process information contains the card ID (Identification) ofthe video processor card 13 _(i). The video processor cards 13 _(i) aresequentially numbered with the card IDs. For example, the videoprocessor cards 13 ₁ through 13 ₆ respectively in the six slots 12 ₁through 12 ₆ are assigned card IDs of 1 from 6, respectively, in thisembodiment.

[0162] Except the video processor card 13 ₁ numbered with #1, which isthe highest in the order of the card IDs, the video processor card 13_(i) numbered #i (i≧2) as the card ID is disabled in operation if thevideo processor card numbered with 13 _(i-1) (i≧2) having the card IDhigher than the video processor card 13 _(i) by one is not loaded.

[0163] For example, to allow the video processor card 13 ₆ numbered withthe #6 card ID to operate, the video processor card 13 ₅ numbered thecard ID #5 must be inserted in the slot thereof. To allow the videoprocessor card 13 ₅ numbered the card ID #5 to operate, the videoprocessor card 13 ₄ numbered with the card ID #4 must be inserted in theslot thereof. Similarly, to allow the video processor card 13 ₆ numberedwith the card ID #6, all video processor cards 13 ₁ through 13 ₅ higherin order than the coefficient generator 13 ₆ must remain inserted intheir respective slots.

[0164] In this embodiment, the process information of the videoprocessor card 13 _(i) having the card ID #i contains the processinformation representing the content of the data conversion processcarried out by the video processor card 13 _(i), and the content of thedata conversion process carried out by the video processor cards 13_(i-1), 13 _(i-2), . . . , 13 ₁ if all video processor card 13 _(i-1),13 _(i-2), . . . , 13 ₁ higher in order than the video processor card 13_(i) are loaded.

[0165] For example, in this embodiment, the video processor card 13 ₁having the card ID #1 becomes different in the function thereof (thecontent of the data conversion process) from when the video processorcard 13 ₁ only is loaded to when both the video processor cards 13 ₁ and13 ₂ are loaded. When the video processor card 13 ₁ only is loaded, thecontent of the data conversion process thereof is defined in the processinformation of the video processor card 13 ₁. When both the videoprocessor cards 13 ₁ and 13 ₂ are loaded, the content of the dataconversion process of the video processor card 13 ₁ is defined in theprocess information of the video processor card 13 ₂. Furthermore, whenboth the video processor cards 13 ₁ and 13 ₂ are loaded, the content ofthe data conversion process of the video processor card 13 ₂ is definedin the process information of the video processor card 13 ₂.

[0166] Similarly, when the video processor cards 13 ₁ through 13 _(i)are loaded, the content of the data conversion process of the videoprocessor cards 13 ₁ through 13 _(i) is defined in the processinformation of the video processor card 13 _(i) having the largest IDnumber (lowest in order).

[0167] In this embodiment, the content of the data conversion process ofthe video processor cards 13 ₁ through 13 _(i) when the video processorcards 13 ₁ through 13 _(i) are loaded are recognized by referencing theprocess information of the video processor card 13 _(i) having thelargest card ID.

[0168] The card controller 98 controls the exchange of the video dataand other data with the card interface 83 (see FIG. 5). The cardcontroller 98 controls the tap extractors 91 and 92, and classclassifier 93. Furthermore, the card controller 98 reads the processinformation from the tap generation storage unit 97 in response to arequest from the video processing interface 40, and then feeds theprocess information to the video processing interface 40. The cardcontroller 98 supplies the coefficient generator 96 with parameters tobe discussed later.

[0169] The coefficient normalizer 99 normalizes the tap coefficientsupplied from the coefficient generator 96 and then feeds the normalizedtap coefficient to the coefficient memory 94.

[0170] Referring to a flow diagram shown in FIG. 7, the process of thevideo processing interface 40 shown in FIG. 5 will be discussed now.

[0171] In step S1, the interface controller 81 in the video processinginterface 40 determines whether the video processor card 13 _(i) isnewly mounted (loaded) in any slot 12 _(i) of the slots 12 ₁ through 12₆ based on the output of the connection detector 84.

[0172] If it is determined in step S1 that the video processor card 13_(i) is loaded in the slot 12 _(i), in other words, if the connectiondetector 84 detects a change in the terminal voltage of the slot 12 _(i)in response of the loading of the video processor card 13 _(i) in theslot 12 _(i), the algorithm proceeds to step S3 skipping step S2.

[0173] If it is determined that the video processor card 13 _(i) is notloaded in the slot 12 _(i), the algorithm proceeds to step S2. Theinterface controller 81 determines whether the video processor card 13_(i) inserted in the slot 12 _(i) of the slots 12 ₁ and 12 ₆ is unloadedbased on the output of the connection detector 84.

[0174] If it is determined in step S2 that no video processor card 13_(i) is unloaded from the slot 12 _(i), the algorithm loops to step S1to repeat the above process.

[0175] If it is determined in step S2 that the video processor card 13_(i) is unloaded from the slot 12 _(i), in other words, if it isdetermined that the connection detector 84 detects a change in theterminal voltage of the slot 12 _(i) in response to the unloading of thevideo processor card 13 _(i) from the slot 12 _(i), the algorithmproceeds to step S3. The interface controller 81 controls the cardinterface 83, thereby reading the process information from all videoprocessor cards 13 loaded in slots 12 ₁ through 12 ₆ (hereinafterreferred to as loaded cards).

[0176] A control signal for requesting the video processor card 13 _(i)to transmit the process information is fed to the card controller 98 inthe video processor card 13 _(i) (see FIG. 6) through the slot 12 _(i).Upon receiving the control signal requesting the process information,the card controller 98 reads the process information from the tapgeneration storage unit 97, and feeds the process information to thecard interface 83. The card interface 83 receives the processinformation supplied from the video processor card 13 _(i), and feedsthe process information to the interface controller 81.

[0177] The algorithm proceeds to step S4. The interface controller 81checks the card ID contained in the process information read from thevideo processor card 13 as a loaded card, and determines the loaded cardto be operative (hereinafter also referred to as an effective card).

[0178] The interface controller 81 arranges all card IDs of the videoprocessor cards 13 as the loaded cards in ascending order. If the cardIDs are consecutive from 1 to n, the (loaded) video processor cards 13 ₁through 13 _(n) respectively having the card IDs of from 1 through nbecome effective cards.

[0179] If the video processor card 13 ₁ is not loaded in the card ID #1even with video processor cards 13 loaded in a plurality of slots of theslots 12 ₁ through 12 ₆, no video processor cards 13 are effective. Evenif the video processor card 13 ₁ having the card ID #1 and the videoprocessor card 13 ₃ having the card ID #3 are loaded in their respectiveslots with the video processor card 13 ₂ having the card ID #2 unloaded,only the video processor card 13 ₁ having the card ID #1 becomeseffective. The video processor card 13 ₃ having the card ID #3 is not aneffective card.

[0180] If video processor cards 13 having the same card ID are loaded ina plurality of slots of the slots 12 ₁ through 12 ₆, the interfacecontroller 81 selects one of the video processor cards 13 having thesame card ID, thereby rendering the selected one effective.

[0181] If a plurality of video processor cards 13 having the same cardID are loaded or if video processor cards 13 having non-consecutivecards IDs are loaded, the interface controller 81 supplies thecontroller 37 with a control signal to that effect. In this case, thecontroller 37 presents a predetermined message on the CRT 11 through theselector 34, frame memory 35, and matrix converter 36. The user is thusnotified that a plurality of video processor cards 13 having the samecard ID are loaded or that video processor cards 13 havingnon-consecutive card IDs are loaded.

[0182] The algorithm proceeds to step S5 after the interface controller81 determines the effective cards in step S4 as discussed above. Theinterface controller 81 controls the card interface 83, therebytransmitting the process information read from the video processor card13 _(i(max)) having the largest card ID among the effective videoprocessor cards 13 _(i) (hereinafter referred to as the maximum IDprocess information) to the card controllers 98 (see FIG. 6) of each ofthe effective video processor cards 13 _(i). In this way, the interfacecontroller 81 controls the tap structures of the predictive tap and theclass tap, the class classification method, and the generation of thetap coefficient in each of the effective video processor cards 13 _(i).

[0183] Each of the effective video processor cards 13 _(i) referencesthe maximum ID process information supplied from (the interfacecontroller 81 in) the video processing interface 40, and recognizes thecontent of the data conversion process which own video processor card 13_(i) must perform. The effective video processor card 13 _(i) generatesthe predictive tap and the class tap having the tap structure based onthe maximum ID process information, performs class classificationaccording to a predetermined class classification method, and generatesa predetermined tap coefficient.

[0184] In step S6, the interface controller 81 determines an input andoutput route of video data for the effective video processor card 13,and sets the card interface 83 to transfer the video data in accordancewith the input and output route. The algorithm loops to step S1.

[0185] The interface controller 81 determines the input and output routeso that the video data read from the frame memory 35 is transferred tothe effective video processor cards 13 in the order of the card IDs. Forexample, if the video processor cards 13 ₁ through 13 ₆ having the cardIDs #1 through #6 are respectively loaded in all slots 12 ₁ through 12 ₆connected to the card interface 83, the input and output routes aredetermined so that the video data is transferred in the order of videoprocessor cards 13 ₁, 13 ₂, 13 ₃, 13 ₄, 13 ₅, and 13 ₆.

[0186] The video data is transferred in the input and output routesdetermined by the card interface 83. If the input and output routes aredetermined so that the video data is transferred to the video processorcards 13 in the order of the video processor cards 13 ₁, 13 ₂, 13 ₃, 13₄, 13 ₅, and 13 ₆, the card interface 83 transfers, to the videoprocessor card 13 ₁, the video data which is read from the frame memory35 by the memory interface 82 and is then supplied through the interfacecontroller 81. Upon receiving the video data, which is data converted bythe video processor card 13 ₁, the card interface 83 transfers the videodata to the video processor card 13 ₂. Likewise, the card interface 83transfers the video data in the order of the video processor cards 13 ₃,13 ₄, 13 ₅, and then 13 ₆. When the card interface 83 receives the videodata from the video processor card 13 ₆, the card interface 83 feeds thevideo data to the interface controller 81. The video data supplied tothe interface controller 81 is stored in the frame memory 35 through thememory interface 82 and line-by-line converter 85, and is then presentedon the CRT 11.

[0187] As discussed above, the video data is transferred to theeffective video processor cards 13 only. The video processor cards 13which are not set to be effective do not process the video data (becausethe video data is not transferred thereto)

[0188] The data conversion process of the video processor card 13 _(i)shown in FIG. 6 is discussed with reference to a flow diagram shown inFIG. 8.

[0189] In step S11, the card controller 98 determines whether theprocess information transmitted from the interface controller 81 (seeFIG. 5) in step S5 in FIG. 7 is received.

[0190] The process information transmitted from the interface controller81 in step S5 in FIG. 7 is the maximum ID process information, andcontains the content of the data conversion process to be carried out bythe video processor card 13 _(i) which is an effective card.

[0191] If it is determined in step S11 that the process information hasbeen received, in other words if it is determined in step S11 that anyof the slots 12 ₁ through 12 ₆ is newly loaded, or that any of the slots12 ₁ through 12 ₆ is unloaded, the algorithm proceeds to step S12. Whenthe process information has been received, the video processor card 13having the maximum card ID changes, and the process information (themaximum ID process information) stored in the video processor card 13having the maximum ID is transmitted to the card controller 98 from (thecard interface 83 in) the video processing interface 40, and the cardcontroller 98 receives the process information. In step S12, the cardcontroller 98 references the maximum ID process information, therebyrecognizing the content of the data conversion process which must becarried by own video processor card 13. Depending on the result of therecognition, the card controller 98 controls the tap extractors 91 and92, and class classifier 93.

[0192] The card controller 98 sets an operation mode in the tapextractor 91 or 92 to construct the predictive tap or the class taphaving a tap structure described in the maximum ID process information.The card controller 98 sets an operation mode of the class classifier 93to perform class classification in accordance with a classclassification method described in the maximum ID process information.

[0193] In step S13, the card controller 98 controls the coefficientgenerator 96 based on the maximum ID process information, therebygenerating a tap coefficient from the tap generation information.

[0194] The process information contains information concerning the tapcoefficient to be generated from the tap generation information(hereinafter referred to as tap coefficient information). Based on thetap coefficient information, the card controller 98 controls thecoefficient generator 96 to generate the tap coefficient from the tapgeneration information. Under the control of the card controller 98, thecoefficient generator 96 generates the tap coefficient for each classfrom the tap generation information stored in the tap generation storageunit 97, and then feeds the tap coefficient to the coefficientnormalizer 99. The card controller 98 then supplies the coefficientgenerator 96 with a parameter to be discussed later the coefficientgenerator 96 requires to generate the tap coefficient from the tapgeneration information.

[0195] When the coefficient generator 96 generates the tap coefficientfor each class and feeds the tap coefficient to the coefficientnormalizer 99, the coefficient normalizer 99 performs a normalizationprocess, for level adjustment to be discussed later, on the tapcoefficients for each class in step S14. The tap coefficient for eachclass subsequent to the normalization process is fed to the coefficientmemory 94 for storage. The algorithm then proceeds to step S15.

[0196] In step S15, the tap extractor 91 sets, as first video data to besubjected to the data conversion process, the video data supplied fromthe card interface 83 (see FIG. 5) through the input and output routeset in step S6 as shown in FIG. 7, and successively sets, as a targetpixel, each of pixels constituting second video data (the video dataobtained subsequent to the data conversion process) corresponding to thefirst video data. The tap extractor 91 extracts a pixel of the firstvideo data with a predictive tap having the tap structure set in stepS12. In step S15, the tap extractor 92 extracts a pixel of the firstvideo data with a class tap having the tap structure set in step S12 forthe target pixel. The predictive tap is fed to the predictor 95 from thetap extractor 91 and the class tap is fed from the tap extractor 92 tothe class classifier 93.

[0197] The class classifier 93 receives the class tap of the targetpixel from the tap extractor 92. In step S16, the class classifier 93classifies the target pixel based on the class tap using the classclassification method set in step S12. Furthermore, the class classifier93 outputs, to the coefficient memory 94, a class code representing theclass of the target pixel obtained as a result of the classclassification. The algorithm then proceeds to step S17.

[0198] In step S17, the coefficient memory 94 reads the tap coefficientstored at an address corresponding to the class code supplied from theclass classifier 93, namely, the tap coefficient of the classcorresponding to the class code, and outputs the tap coefficient. Alsoin step S17, the predictor 95 acquires the tap coefficient output fromthe coefficient memory 94. The algorithm then proceeds to step S18.

[0199] In step S18, the predictor 95 performs a predetermined predictioncalculation (prediction calculation represented by equation (1) to bediscussed later) using the predictive tap output from the tap extractor91 and the tap coefficient retrieved from the coefficient memory 94. Thepredictor 95 thus determines a pixel value of the target pixel, andfeeds the pixel value to the card interface 83 (see FIG. 5). Thealgorithm loops to step S11 to repeat the same process.

[0200] If it is determined in step S11 that the process information hasnot been received, the algorithm then proceeds to step S15, skippingsteps S12 through S14. The same process as described above will then berepeated. The process described in steps S15 through S18 is carried outin accordance with the process information (the maximum ID processinformation) previously received by the card controller 98.

[0201] Discussed next are the prediction calculation of the predictor95, the generation of the tap coefficient by the coefficient generator96, and the learning process of coefficient seed data, as one piece ofthe tap generation information stored in the tap generation storage unit97 illustrated in FIG. 6.

[0202] Now, let the second video data be high-quality video data, andlet the first video data be low-quality video data which is obtained bydegrading the image quality (resolution) of the high-quality video data,for example, by filtering the high-quality video data through an LPF(Low-Pass Filter). The predictive tap is extracted from the low-qualitydata, and the pixel value of the high-quality pixel is determined(predicted) using a predetermined prediction calculation based on thepredictive tap and predetermined tap coefficient.

[0203] If the predetermined prediction calculation is a first orderlinear prediction calculation, the pixel value y of the high-qualitypixel is determined using the following first order equation.$\begin{matrix}{y = {\sum\limits_{n = 1}^{N}\quad {w_{n}x_{n}}}} & (1)\end{matrix}$

[0204] where x_(n) represents a pixel value of a pixel of n-thlow-quality video data forming a predictive tap for a high-quality pixely, and w_(n) is an n-th tap coefficient which is to be multiplied by(the pixel value of) the n-th low-quality pixel.

[0205] In equation (1), the predictive tap includes N low-quality pixelsx₁, x₂, . . . , x_(N). In accordance with equation (1), the high-qualitypixel y is determined by weighting N low-quality pixels x₁, x₂, . . . ,x_(N) respectively with N tap coefficients w₁, w₂, . . . , w_(N), and bysumming the weighted low-quality pixels. To prevent a level variation inthe high-quality pixel y determined by equation (1), the sum of the tapcoefficients, namely, weights, W₁, w₂, . . . , w_(N) must be 1. Thecoefficient normalizer 99 in the video processor card 13 (see FIG. 6)sums the tap coefficients for each class supplied from the coefficientgenerator 96. Each tap coefficient is normalized by dividing each tapcoefficient of the class by the sum of the tap coefficients.

[0206] The pixel value y of the high-quality pixel may be determined bysecond order or higher order equation rather than the first order linearequations in equation (1).

[0207] Let y_(k) represent the true pixel value of a high-quality pixelof a k-th sample, and y_(k)′ represent the predictive value of the truevalue y_(k) determined from equation (1), and a predictive error e_(k)is expressed by equation (2).

e _(k) =y _(k) −y _(k)′  (2)

[0208] The predictive value Y_(k)′ in equation (2) is determined fromequation (1). If the predictive value Y_(k)′ determined by equation (1)is substituted in equation (2), equation (3) results. $\begin{matrix}{e_{k} = {y_{k} - \left( {\sum\limits_{n = 1}^{N}\quad {w_{n}x_{n,k}}} \right)}} & (3)\end{matrix}$

[0209] In equation (3), x_(n,k) represents n-th low-quality pixelforming the predictive tap of the high-quality image at the k-th sample.

[0210] A tap coefficient w_(n) to cause the predictive error e_(k) to bezero in equation (3) is optimum for predicting the high-quality image.Generally, it is difficult to determine such a tap coefficient W_(n) forall high-quality pixels.

[0211] A measure of an optimum tap coefficient w_(n) may be based onleast square method, for example. The optimum tap coefficient w_(n) isobtained by minimizing a sum E of the squared errors. $\begin{matrix}{E = {\sum\limits_{k = 1}^{K}\quad e_{k}^{2}}} & (4)\end{matrix}$

[0212] where K represents the number of samples (the number of learningsamples) of sets, each set including the high image quality pixel y_(i)and the predictive tap of the low image quality pixel formed of x_(1,k),x_(2,k), . . . , x_(N,k) with respect to the high image quality pixely_(i).

[0213] The tap coefficient w_(n) for minimizing the sum E of the squarederrors causes the partial differentiation of the sum E with the tapcoefficient w_(n) to be zero. Equation (5) thus holds. $\begin{matrix}{\frac{\partial E}{\partial w_{n}} = {{{e_{1}\frac{\partial e_{1}}{\partial w_{n}}} + {e_{2}\frac{\partial e_{2}}{\partial w_{n}}} + \ldots + {e_{k}\frac{\partial e_{n}}{\partial w_{n}}}} = {0\quad \left( {{n = 1},2,\quad \ldots \quad,N} \right)}}} & (5)\end{matrix}$

[0214] If the above-referenced equation (3) is partially differentiatedwith respect to the tap coefficient w_(n), equation (6) results.$\begin{matrix}{{\frac{\partial e_{k}}{\partial w_{1}} = {- x_{1,k}}},{\frac{\partial e_{k}}{\partial w_{2}} = {- x_{2,k}}},\quad \ldots \quad,{\frac{\partial e_{k}}{\partial w_{N}} = {- x_{N,k}}},\left( {{k = 1},2,\quad \ldots \quad,K} \right)} & (6)\end{matrix}$

[0215] From equations (5) and (6), equation (7) results. $\begin{matrix}{{{\sum\limits_{k = 1}^{K}\quad {e_{k}x_{1,k}}} = 0},{{\sum\limits_{k = 1}^{K}\quad {e_{k}x_{2,k}}} = 0},\quad {{\ldots \quad {\sum\limits_{k = 1}^{K}\quad {e_{k}x_{N,k}}}} = 0}} & (7)\end{matrix}$

[0216] By substituting equation (3) for e_(k) in equation (7), equation(7) becomes normal equation (8). $\begin{matrix}\begin{matrix}{{{\begin{bmatrix}\left( {\sum\limits_{k = 1}^{K}\quad {x_{1,k}x_{1,k}}} \right) & \left( {\sum\limits_{k = 1}^{K}\quad {x_{1,k}x_{2,k}}} \right) & \cdots & \left( {\sum\limits_{k = 1}^{K}\quad {x_{1,k}x_{N,k}}} \right) \\\left( {\sum\limits_{k = 1}^{K}\quad {x_{2,k}x_{1,k}}} \right) & \left( {\sum\limits_{k = 1}^{K}\quad {x_{2,k}x_{2,k}}} \right) & \cdots & \left( {\sum\limits_{k = 1}^{K}\quad {x_{2,k}x_{N,k}}} \right) \\\vdots & \vdots & ⋰ & \vdots \\\left( {\sum\limits_{k = 1}^{K}\quad {x_{N,k}x_{1,k}}} \right) & \left( {\sum\limits_{k = 1}^{K}\quad {x_{N,k}x_{2,k}}} \right) & \cdots & \left( {\sum\limits_{k = 1}^{K}\quad {x_{N,k}x_{N,k}}} \right)\end{bmatrix}\quad\begin{bmatrix}w_{1} \\\quad \\w_{2} \\\quad \\{\vdots \quad} \\\quad \\w_{N}\end{bmatrix}}{\begin{matrix} = \\\quad \\ = \\\quad \\\quad \\\quad \\ = \end{matrix}\begin{bmatrix}\left( {\sum\limits_{k = 1}^{K}\quad {x_{1,k}y_{k}}} \right) \\\left( {\sum\limits_{k = 1}^{K}\quad {x_{2,k}y_{k}}} \right) \\\vdots \\\left( {\sum\limits_{k = 1}^{K}\quad {x_{N,k}y_{k}}} \right)\end{bmatrix}}}\quad} \\\quad\end{matrix} & (8)\end{matrix}$

[0217] Equation (8) may be solved for the tap coefficient w_(n) usingthe sweep method (Gauss-Jordan elimination).

[0218] By writing and solving the normal equation (8) for each class,the optimum tap w_(n) (the tap coefficient that minimizes the sum E ofthe squared errors) is determined for each class.

[0219]FIG. 9 illustrates the construction of a learning device whichperforms a learning process for determining the tap coefficient w_(n)for each class by writing and then solving the normal equation (8).

[0220] The learning device receives education video data for use in thelearning process. The education video data, here corresponding to thesecond video data, is high-resolution video data, for example.

[0221] In the learning device, the education video data is fed to atraining data generator 111 and a learning data generator 113.

[0222] The training data generator 111 generates training data from thesupplied education video data, and feeds the training data to a trainingdata storage unit 112. Specifically, the training data generator 111supplies the training data storage unit 112 with the high-quality videodata as the training data.

[0223] The training data storage unit 112 stores the high-quality dataas the training data supplied from the training data generator 111.

[0224] In response to the education video data, the learning datagenerator 113 generates. learning data as the first video data, andfeeds the learning data to a learning data storage unit 114.Specifically, the learning data generator 113 filters the high-qualityvideo data as the education video data, thereby reducing the resolutionof the high-quality video data. Low-quality video data is thusgenerated, and is fed to the learning data storage unit 114 as thelearning data.

[0225] The learning data storage unit 114 stores the learning datasupplied from the learning data generator 113.

[0226] A tap extractor 115 successively sets each of pixels constitutingthe high-quality video data as the training data stored in the trainingdata storage unit 112 as a target training pixel, and extractspredetermined pixels of low-quality pixels constituting the low-qualityvideo data as the learning data stored in the learning data storage unit114 with respect to the target training pixel. The tap extractor 115thus constructs a predictive tap having the same tap structure as theone constructed by the tap extractor 91 shown in FIG. 6, and feeds thepredictive tap to a factoring unit 118.

[0227] A tap extractor 116 extracts predetermined pixels of low-qualitypixels constituting the low-quality video data as the learning datastored in the learning data storage unit 114 with respect to the targettraining pixel. The tap extractor 116 thus constructs a class tap havingthe same tap structure as the one constructed by the tap extractor 92shown in FIG. 6, and feeds the class tap to a class classifier 117.

[0228] The tap extractors 115 and 116 receive the process informationgenerated by a process information generator 120. The tap extractors 115and 116 respectively construct the predictive tap and the class taphaving the tap structures represented by the process informationsupplied from the process information generator 120.

[0229] The class classifier 117 performs class classification identicalto that performed by the class classifier 93 shown in FIG. 6, based onthe class tap output from the tap extractor 116. A class code obtainedas a result of the class classification is output to the factoring unit118.

[0230] The class classifier 117 receives the process informationgenerated from the process information generator 120. The classclassifier 117 performs class classification using a classclassification method represented by the process information suppliedfrom the process information generator 120.

[0231] The factoring unit 118 reads the target training pixel from thetraining data storage unit 112, and performs, for each class, afactoring operation to the read target training pixel and the learningdata forming the predictive tap which is constructed with respect to thetarget training pixel supplied from the tap extractor 115.

[0232] Specifically, the factoring unit 118 uses the predictive tap(learning data) x_(i,k) (x_(j,k)) for each class corresponding to theclass code supplied from the class classifier 117. The factoring unit118 thus multiplies learning data by learning data (x_(i,k) x_(j,k)) andthen performs a summing operation to the resulting product of learningdata to determine each element in the matrix on the left side ofequation (8).

[0233] Furthermore, the factoring unit 118 uses the predictive tap(learning data) x_(i,k) and the training data y_(k) for each classcorresponding to the class code supplied from the class classifier 117.The factoring unit 118 thus multiplies the learning data x_(i,k) by thetraining data y_(k) and then performs a summing operation to theresulting product (x_(i,k) y_(k)) of the learning data and the trainingdata to determine each element in the vector on the right side ofequation (8).

[0234] Specifically, the factoring unit 118 stores, in a built-in memory(not shown), the previous elements of the matrix on the left side ofequation (8) determined for the training data as the target trainingpixel and the previous elements in the vector on the right side. As forthe training data as a new training pixel, the factoring unit 118factors in the elements x_(i,k)x_(j,k) or x_(i,k)y_(k), which arecalculated using the training data y_(k) and the learning data x_(i,k)(x_(j,k)), as elements of the matrix and elements in the vector.

[0235] The factoring unit 118 performs the factoring operation bytreating all training data stored in the training data storage unit 112as the target pixel, and writes the normal equation (8) for each class,and then feeds the normal equation (8) to a tap coefficient calculator119.

[0236] The tap coefficient calculator 119 solves the normal equation (8)supplied from the factoring unit 118 for each class, thereby determiningand outputting the tap coefficient w_(n) for each class.

[0237] The process information generator 120 generates the processinformation such as the tap structure of the predictive tap and theclass tap constructed by each of the video processor cards 13 ₁ through13 ₆, and the class classification method of the class classificationperformed by each of the video processor cards 13 ₁ through 13 ₆, andfeeds the process information to the tap extractors 115 and 116 andclass classifier 117. The class structure and the class classificationmethod contained in the process information in the process informationgenerator 120 are stored (registered) beforehand in the processinformation generator 120.

[0238] The learning process of the learning device illustrated in FIG. 9is discussed with reference to a flow diagram illustrated in FIG. 10.

[0239] In step S21, the training data generator 111 and the learningdata generator 113 generate and output the training data and learningdata, respectively, in response to the education video data.Specifically, the training data generator 111 directly outputs theeducation video data as the training data. The learning data generator113 generates the learning data, corresponding to the training data (theeducation video data) of each frame (or each field), from the educationvideo data by filtering the education video data through an LPF.

[0240] The training data output from the training data generator 111 isfed to the training data storage unit 112 for storage, and the learningdata output from the learning data generator 113 is fed to the learningdata storage unit 114 for storage.

[0241] In step S22, the process information generator 120 generates thepredetermined process information, and feeds the process information tothe tap extractors 115 and 116, and class classifier 117. The tapstructure of the predictive tap constructed by the tap extractor 115,the tap structure of the class tap constructed by the tap extractor 116,and the class classification method of the class classifier 117 are setin this way.

[0242] In step S23, the tap extractor 115 sets, as a target trainingpixel, training data stored in the training data storage unit 112 andnot yet set as a target. Also in step S23, the tap extractor 115constructs the predictive tap from the learning data stored in thelearning data storage unit 114 in connection with the target trainingpixel, and feeds the predictive tap to the factoring unit 118. The tapextractor 116 constructs the class tap from the learning data stored inthe learning data storage unit 114 in connection with the targettraining pixel, and feeds the class tap to the class classifier 117.

[0243] The tap extractor 115 constructs the predictive tap of having thetap structure set in step S22. The tap extractor 116 constructs theclass tap having the tap structure set in step S22.

[0244] In step S24, the class classifier 117 classifies the targettraining pixel based on the class tap of the target training pixel, andoutputs the class code corresponding to the resulting class to thefactoring unit 118. The algorithm proceeds to step S25.

[0245] The class classifier 117 classifies the target training pixel inaccordance with the class classification method set in step S22.

[0246] In step S25, the factoring unit 118 reads the target trainingpixel from the training data storage unit 112, and calculates theelements x_(i,k)x_(j,k) in the matrix on the left side of equation (8)and the elements x_(i,k)y_(k) in the vector on the right side of theequation (8) using the read target training pixel and the predictive tapsupplied from the tap extractor 115. Furthermore, the factoring unit 118factors in the elements x_(i,k)x_(j,k) in the matrix and the elementsx_(i,k)y_(k) in the vector determined from the target pixel and thepredictive tap to elements corresponding to the class code from theclass classifier 117, out of already obtained elements in the matrix andin the vector. The algorithm proceeds to step S26.

[0247] In step S26, the tap extractor 115 determines whether thetraining data storage unit 112 still stores training data not yet set asa target training pixel. If it is determined in step S26 that thetraining data storage unit 112 still stores training data not yet set asa target training pixel, the tap extractor 115 sets that training dataas a new target training pixel. The algorithm loops to step S23 toperform the same process.

[0248] If it is determined in step S26 that the training data storageunit 112 does not store training data not yet set as a target trainingpixel, the factoring unit 118 supplies the tap coefficient calculator119 with the matrix on the left side and the vector on the right side ofequation (8) obtained for each class from the above process. Thealgorithm then proceeds to step S27.

[0249] In step S27, the tap coefficient calculator 119 solves the normalequation for each class formed of the matrix on the left side and thevector on the right side of equation (8) for each class supplied fromthe factoring unit 118. The tap coefficient calculator 119 thusdetermines and outputs the tap coefficient w_(n) and ends the process.

[0250] A class may occur which has an insufficient number of normalequations required to determine the tap coefficient w_(n) because of aninsufficient number of pieces of education video data. For such a class,the tap coefficient calculator 119 outputs a default tap coefficient,for example. The default tap coefficient may be a tap coefficient whichis determined without performing class classification (with a totalnumber of classes set to 1). In the above case, the education video databecomes the training data as the second video data, and the low-qualityvideo data that is obtained by degrading the education video data inspatial resolution is set to be the learning data as the first videodata. The learning process of the tap coefficient is performed using thefirst video data and second video data. The tap coefficient is obtainedso that the data conversion process is performed as a resolutionenhancement process for converting the first video data to the secondvideo data with the resolution thereof enhanced. By storing the tapcoefficient in the coefficient memory 94 (see FIG. 6) in the videoprocessor card 13, the video processor card 13 enhances the spatialresolution of the video data.

[0251] The tap coefficient performs a variety of data conversionprocesses depending on the manner of selecting the learning data as thefirst video data and the training data as the second video data.

[0252] For example, the learning process is performed with high-qualityvideo data set to be the training data and video data with noisesuperimposed thereof set to be the learning data. With the tapcoefficient, the data conversion process functions as a noise removalprocess for converting the first video data to the second video datawith noise thereof removed (reduced).

[0253] The learning process may performed with video data set to be thetraining data and the video data with the number of pixels reduced beingset to be the learning data. The learning process may be performed withvideo data having a predetermined size set to be the learning data andwith the video data with the pixels thereof reduced at a predetermineddecimation ratio set to be the training data. With the tap efficient,the data conversion process functions as a data resizing process forconverting the first video data to the second video data which is anexpanded version or a contracted version of the first video data.

[0254] The video processor card 13 performs the noise removal process orthe resizing process (for expansion or contraction) by allowing the tapcoefficient for the noise removal process or the tap coefficient for theresizing process to be stored in the coefficient memory 94 (see FIG. 6)in the video processor card 13.

[0255] Depending on a set of the training data and the learning data(hereinafter also referred to an education pair) used for learning thetap coefficient, the tap coefficient achieves a variety of image qualityimprovements such as the enhancement of the resolution, the noiseremoval, or the resizing process. By allowing the video processor cards13 ₁ through 13 ₆ to store the tap coefficient for performing a varietyof image quality improvements, the video processor card 13 _(i) performsthe data conversion process for a variety of image quality improvements.

[0256] Furthermore, the learning process may be performed withhigh-quality video data set to be the training data and with thehigh-quality data with the spatial resolution thereof reduced and withnoise superimposed thereon set to be the learning data. With the tapcoefficient, the data conversion process functions as noise removalprocess and resolution enhancement process, wherein the first video datais converted into the second video data by removing (reducing) noise andby enhancing the spatial resolution thereof.

[0257] The video data may be converted using the tap coefficient forperforming both the noise removal process and the resolution enhancementprocess. The video data may be converted using the tap coefficient forthe noise removal process, and then the converted video data may befurther converted using the tap coefficient for the resolutionenhancement process. Given the same data conversion conditions, thesetwo pieces of resulting video data become different in image quality.

[0258] The predictive tap, and the class tap having the same tapstructure, and the same class classification method (with the samenumber of classes) may be used in each of the data conversion processusing the tap coefficient for both the noise removal and the resolutionenhancement, the data conversion process using the tap coefficient forthe noise removal, and the data conversion process using the tapcoefficient for the resolution enhancement. Under these conditions, thevideo data resulting from the data conversion process using the tapcoefficient for the noise removal and then the data conversion processusing the tap coefficient for the resolution enhancement is better inimage quality than the video data resulting from the data conversionprocess using the tap coefficient for both the noise removal and theresolution enhancement.

[0259] The tap generation storage unit 97 in the video processor card 13can store, as the tap generation information, the tap coefficient foreach class determined in the learning device shown in FIG. 9.

[0260] Let Z represent the number of classes, and let B represent thesize of a set of tap coefficients per class (tap coefficients w₁, w₂, .. . , w_(N) defining equation (1)), and the tap generation storage unit97 forming the video processor card 13 requires at least a memorycapacity of Z×B.

[0261] Let subscript n be a tap number of a n-th tap coefficient w_(n)of the set of tap coefficients w₁, w₂, . . . , w_(N), and the tapcoefficients w₁ through w_(N) for each class are compressed on a per tapnumber basis. The compressed tap coefficients may be stored in the tapgeneration storage unit 97 (see FIG. 6) as the tap generationinformation. The required memory capacity of the tap generation storageunit 97 is thus reduced.

[0262] The compression of the tap coefficients w₁ through w_(N) for eachclass on a per tap number basis may be performed as below.

[0263] The tap coefficient w_(n) is now constructed of coefficient seeddata serving as a seed for the tap coefficient and a predeterminedparameter in accordance with equation (9). $\begin{matrix}{w_{n} = {\sum\limits_{m = 1}^{M}\quad {\beta_{m,n}z^{m - 1}}}} & (9)\end{matrix}$

[0264] where β_(m,n) represents m-th coefficient seed data used todetermine an n-th tap coefficient w_(n), and z represents a parameter.In equation (9), the tap coefficient w_(n) is determined using M piecesof coefficient seed data β_(1,n), β_(2,n), . . . , β_(M,n).

[0265] Equations for determining the tap coefficient w_(n) from thecoefficient seed data β_(m,n) and the parameter z are not limited toequation (9).

[0266] A value z^(m-1) determined by the parameter z in equation (9) isdefined by equation (10) as below by introducing a new variable t_(m).

t _(m) =z ^(m-1) (m=1, 2, . . . , M)  (10)

[0267] Equation (11) may be obtained by combining equations (9) and(10). $\begin{matrix}{w_{n} = {\sum\limits_{m = 1}^{M}\quad {\beta_{m,n}t_{m}}}} & (11)\end{matrix}$

[0268] According to equation (11), the tap coefficient w_(n) isdetermined (predicted) from a linear first-order equation of thecoefficient seed data β_(m,n) and the variable t_(m). Let w_(n)′represent the tap coefficient determined from equation (11), and let anerror e_(n) represent a difference between an appropriate tapcoefficient w_(n) and the tap coefficient determined w_(n)′ fromequation (11), and the coefficient seed data β_(m,n) causing the errore_(n) to be zero becomes appropriate for determining the appropriatew_(n). It is generally difficult to determine the coefficient seed dataβ_(m,n) for all tap coefficients w_(n).

e _(n) =w _(n) −w _(n)′  (12)

[0269] Equation (12) is rewritten by substituting equation (11) forw_(n)′ on the right side as below. $\begin{matrix}{e_{n} = {w_{n} - \left( {\sum\limits_{m = 1}^{M}\quad {\beta_{m,n}t_{m}}} \right)}} & (13)\end{matrix}$

[0270] A measure of an optimum coefficient seed data β_(m,n) may bebased on least square method, for example. The optimum coefficient seeddata β_(m,n) is obtained by minimizing a sum E of the squared errorsexpressed by equation (14). $\begin{matrix}{E = {\sum\limits_{n = 1}^{N}\quad e_{n}^{2}}} & (14)\end{matrix}$

[0271] The minimum value of the sum E of the squared error expressed byequation (14) results from β_(m,n) which causes partial differentiationof the sum E with respect to the coefficient seed data β_(m,n) to zeroas expressed by equation (15). $\begin{matrix}{\frac{\partial E}{\partial\beta_{m \cdot n}} = {{\sum\limits_{m = 1}^{M}\quad {2{\frac{\partial e_{n}}{\partial\beta_{m \cdot n}} \cdot e_{n}}}} = 0}} & (15)\end{matrix}$

[0272] Equation (16) is obtained by combining equations (13) and (15).$\begin{matrix}{{\sum\limits_{m = 1}^{M}{t_{m}\left( {w_{n} - \left( {\sum\limits_{m = 1}^{M}{\beta_{m \cdot n}t_{m}}} \right)} \right)}} = 0} & (16)\end{matrix}$

[0273] Now, X_(i,j) and Y_(i) are respectively defined by equations (17)and (18). $\begin{matrix}{X_{i,j} = {\sum\limits_{z = 1}^{Z}\quad {t_{i}t_{j}\quad \left( {{i = {1,2}},\quad \ldots \quad,{{M:j} = {1,2}},\quad \ldots \quad,M} \right)}}} & (17) \\{Y_{i} = {\sum\limits_{z = 1}^{Z}\quad {t_{i}w_{n}}}} & (18)\end{matrix}$

[0274] Equation (16) is expressed by normal equation (19) using X_(i,j)and Y_(i). $\begin{matrix}{{\begin{bmatrix}X_{1,1} & X_{1,2} & \cdots & X_{1,M} \\X_{2,1} & X_{2,1} & \cdots & X_{2,2} \\\vdots & \vdots & ⋰ & \vdots \\X_{M,1} & X_{M,2} & \cdots & X_{M,M}\end{bmatrix}\begin{bmatrix}\beta_{1,n} \\\beta_{2,n} \\\vdots \\\beta_{M,n}\end{bmatrix}} = \begin{bmatrix}Y_{1} \\Y_{2} \\\vdots \\Y_{M}\end{bmatrix}} & (19)\end{matrix}$

[0275] The normal equation (19) may be solved for the coefficient seeddata β_(m,n) using the sweep method (Gauss-Jordan elimination).

[0276] The coefficient seed data β_(m,n) determined by solving thenormal equation (19) results in the tap coefficient w_(n) for eachparameter z in accordance with equation (9).

[0277] When the class of the tap coefficient is used as the parameter z,a set of tap coefficients for class #z is determined based on theparameter (class) z and the coefficient seed data β_(m,n) using equation(9).

[0278] Let w^((z)) _(n) represent an n-th tap coefficient w_(n) in class#z, and the coefficient seed data β_(m,n) for determining the n-th tapcoefficient x^((z)) _(n) in the class #z using equation (9) isdetermined by writing and solving the normal equation (19). In thiscase, the n-th tap coefficients w⁽¹⁾ _(n), w⁽²⁾ _(n), . . . , w^((z))_(n) in respective classes #1, #2, . . . , #z determined by the learningdevice shown in FIG. 9 are respective training data, and the parameters#1, #2, . . . , #Z representing classes are learning data correspondingto the training data for respective classes.

[0279]FIG. 11 shows the construction of a learning device that performsa learning process to determine the coefficient seed data β_(m,n) fordetermining the n-th tap coefficient w^((z)) _(n) in the class #z. Asshown, components identical to those described with reference to FIG. 9are designated with the same reference numerals and the discussionthereof is omitted here.

[0280] A tap coefficient memory 121 stores a set of tap coefficients foreach class output from the tap coefficient calculator 119.

[0281] A factoring unit 122 factors in (the variable t_(m) correspondingto) the parameter #z representing the class, and the tap coefficientw^((z)) _(n) of each class #z stored in the tap coefficient memory 121with respect to each tap number n.

[0282] The factoring unit 122 performs a factoring operation using avariable t_(i)(t_(j)) determined from the parameter z using equation(10). Specifically, variables t_(i)(t_(j)) for the parameter z todetermine an element X_(i,j), defined by equation (17), in the matrix onthe left side of equation (19) are multiplied by each other, and theresulting products are summed for each tap number n.

[0283] Furthermore, the factoring unit 122 performs a factoringoperation using the variable t_(i) determined from equation (10) and thetap coefficient w^((z)) _(n). Specifically, the variable t_(i)respectively corresponding to the parameter z for determining Y_(i)defined by equation (18) is multiplied by the tap coefficient w_(n) andthe resulting products (t_(i)w_(n)) are summed for each tap number n.Y_(i) is an element in the vector on the right side of equation (19).

[0284] The factoring unit 122 determines the element X_(i,j) expressedby equation (17) and the element Y_(i) expressed by equation (18) foreach tap number, thereby writing the normal equation (19) for each tapnumber n. The factoring unit 122 then outputs the normal equation to acoefficient seed calculator 123.

[0285] In the embodiment illustrated in FIG. 11, the process informationoutput from the process information generator 120 is fed to thefactoring unit 122. In this case, the process information contains Mrepresenting the number of terms in equation (9) as a tap calculationequation for determining the tap coefficient w^((z)) _(n) from thecoefficient seed data β_(m,n) The factoring unit 122 references theprocess information supplied from the process information generator 120,thereby recognizing the number of terms M, and writing the normalequation (19).

[0286] The coefficient seed calculator 123 determines and outputs thecoefficient seed data β_(m,n) for each tap number by solving the normalequation (19) for each tap number n supplied from the factoring unit122.

[0287] The learning process of the learning device illustrated in FIG.11 is discussed below with reference to a flow diagram illustrated inFIG. 12.

[0288] Steps S31 through S37 perform the same process as steps S21through S27 shown in FIG. 10. The tap coefficients determined by the tapcoefficient calculator 119 for each class are fed to the tap coefficientmemory 121 for storage.

[0289] Set in step S32 are the tap structure of the predictive tapconstructed by the tap extractor 115, the tap structure of the class tapconstructed by the tap extractor 116, the class classification method ofthe class classifier 117, and the number of terms M of the tapcalculation equation (9) required for the factoring unit 122 to writethe normal equation (19).

[0290] The process information generator 120 in the learning deviceshown in FIG. 11 generates the process information including the tapstructures of the predictive tap and the class tap constructed by eachof the video processor cards 13 ₁ through 13 ₆, the class classificationmethod of the class classification performed by each of the videoprocessor cards 13 ₁ through 13 ₆, and the number of terms M of the tapcalculation equation (9) for generating the tap coefficient w_(n) fromthe coefficient seed data β_(m,n) in each of the video processor cards13 ₁ through 13 ₆. The process information generator 120 then feeds theprocess information to the tap extractors 115 and 116, class classifier117, and factoring unit 122. In accordance with the number of terms Mcontained in the process information, the factoring unit 122 sets theoperation mode thereof to write the normal equation (19).

[0291] In step S38, the factoring unit 122 performs a factoringoperation to (the variable t_(m) corresponding to) the parameter #zrepresenting the class and the tap coefficient w^((z)) _(n) of eachclass #z stored in the tap coefficient memory 121 for each tap number n.The factoring unit 122 thus writes the normal equation (19) and feedsthe normal equation (19) to the coefficient seed calculator 123. Thealgorithm then proceeds to step S39.

[0292] In step S38, the factoring unit 122 writes the normal equation(19) in accordance with the number of terms M of the tap calculationequation (9) set in step S32.

[0293] In step S39, the coefficient seed calculator 123 solves thenormal equation (19) for each tap number n supplied from the factoringunit 122, thereby determining and outputting the coefficient seed dataβ_(m,n) for each tap number n. The process ends.

[0294] The learning device shown in FIG. 11 modifies the education pairand the process information as appropriate, and performs the learningprocess shown in FIG. 12. In this way, the coefficient seed data β_(m,n)for performing a variety of image quality improvements is thusdetermined. Different coefficient seed data β_(m,n) for performingdifferent image improvements are respectively stored in the videoprocessor cards 13 ₁ through 13 ₆.

[0295] Assuming that the number of the tap coefficients w_(n) for oneclass is N, and that the number of classes is Z, the total number of tapcoefficients w_(n) is N×Z.

[0296] Assuming that the number of the tap coefficients w_(n) for oneclass is N, and that the number of terms of the tap calculation equation(9) is M, the total number of coefficient seed data β_(m,n) is N×M.

[0297] If a single tap coefficient w_(n) has the same amount of data asthe coefficient seed data β_(m,n) the amount of data of all coefficientseed data β_(m,n) is M/Z times the amount of data of all tapcoefficients w_(n).

[0298] Now, M is compared with Z. M is the number of terms in the tapcalculation equation (9), and is typically one digit number or two digitnumber (10-99). On the. other hand, Z is the number of classes. Theclass classification for determining the class is performed by ADRCprocessing the class tap. For example, if a class tap composed of 25(5×5) pixels is classified by one-bit ADRC processing, the number ofclasses is 2²⁵ classes.

[0299] The value of M/Z is sufficiently small, and the tap coefficientw_(n) is compressed to the coefficient seed data β_(m,n) having asufficiently small amount of data.

[0300] The coefficient seed data β_(m,n) is stored as the tap generationinformation in the tap generation storage unit 97 (see FIG. 6). Incomparison with the case in which the tap coefficient w_(n) is stored,the required memory capacity of the tap generation storage unit 97 issubstantially reduced.

[0301] The process information stored in the video processor card 13_(i) shown in FIG. 6 contains, besides the tap structure and the classclassification method, the number of terms M of the tap calculationequation (9), and the number of classes Z in the class classification ofthe class classifier 93. The coefficient generator 96 references theprocess information, recognizing the number of terms M of the tapcalculation equation (9). The card controller 98 references the processinformation, thereby recognizing the number of classes Z. The cardcontroller 98 feeds an integer from 1 through Z as the parameter z tothe coefficient generator 96. The coefficient generator 96 uses thecoefficient seed data β_(m,n) as the process information and theparameter z supplied from the card controller 98, thereby summing the Mterms on the right side of the tap calculation equation (9). The tapcoefficient w_(n) for each class is thus determined.

[0302] As described above, the learning device shown in FIG. 11 sets thehigh-quality video data to be the training data and sets the low-qualityvideo data with the image quality thereof degraded to be the learningdata. Sets of tap coefficients for Z classes are generated. The learningdevice shown in FIG. 11 compresses the tap coefficients of the Zclasses, thereby generating a set of coefficient seed data of N taps.The number of predictive taps is N.

[0303] Depending on what learning pair (of the training data and thelearning data) is used, the coefficient seed data forms the tapcoefficient for different types of image quality improvements (such asnoise reduction or resolution enhancement) in the learning process.Furthermore, the coefficient seed data forms the tap coefficientresulting in different image quality improvements, depending on the tapstructure of the predictive tap (the position of a pixel serving as thepredictive tap, and the number of taps) used in the learning process,the tap structure of the class tap, the class classification method, andthe number of terms M of the tap calculation equation (9).

[0304] The video processor cards 13 ₁ through 13 ₆ store, as the tapgeneration information, the coefficient seed data for generating the tapcoefficient w_(n) for image quality improvements in type and/or level.The coefficient seed data stored in the video processor cards 13 ₁ and13 ₂, out of the video processor card 13 ₁ through 13 ₆, is discussedwith reference to FIG. 14 through FIG. 19.

[0305] Referring to FIG. 14, the video processor card 13 ₁ stores a setof 9 tap coefficient seed data as the tap generation information. Thehigh-quality video data is set to be the training data. The low-qualityvideo data is obtained by degrading the high-quality video data inresolution and then by adding noise to the high-quality video data. Thelearning process is performed with the high-quality video data as thetraining data and with the low-quality video data as the learning data.A set of tap coefficients for 64 classes for resolutionenhancement/noise removal for enhancing resolution and removing noise isgenerated, and then compressed, becoming the set of 9 tap coefficientseed data for resolution enhancement/noise removal.

[0306] Referring to FIG. 15, the video processor card 13 ₁ stores a setof 25 tap coefficient seed data as the tap generation information inaddition to the set of 9 tap coefficient seed data for resolutionenhancement/noise removal shown in FIG. 14. The high-quality video datais set to be the training data. The low-quality video data is obtainedby degrading the high-quality video data in resolution. The learningprocess is performed with the high-quality video data as the trainingdata and with the low-quality video data as the learning data. A set oftap coefficients for 64 classes for resolution enhancement is generatedand then compressed, becoming the set of 25 tap coefficient seed datafor resolution enhancement.

[0307] Referring to FIG. 16, the video processor card 13 ₂ stores a setof 25 tap coefficient seed data as the tap generation information. Thehigh-quality video data is set to be the training data. The low-qualityvideo data is obtained by adding noise to the high-quality video data.The learning process is performed with the high-quality video data asthe training data and with the low-quality video data as the learningdata. A set of tap coefficients for 64 classes for resolutionenhancement is generated, and then compressed, becoming the set of 25tap coefficient seed data for noise removal.

[0308] If the video processor card 13 ₁ only is loaded in the videoprocessing interface 40, the video processor card 13 ₁ generates the setof tap coefficients for 64 classes for resolution enhancement/noiseremoval from the set of 9 tap coefficient seed data for resolutionenhancement/noise removal, out of the set of 9 tap coefficient seed datafor resolution enhancement/noise removal and the set of 25 tapcoefficient seed data for resolution enhancement, as shown in FIG. 17.Using the set of tap coefficients for the 64 classes for resolutionenhancement/noise removal, the video processor card 13 ₁ performs thedata conversion process on the first video data supplied from the framememory 35 (see FIG. 3), thereby outputting the second video data withthe resolution thereof enhanced and the noise thereof removed.

[0309] The process information stored in the tap generation storage unit97 (see FIG. 6) in the video processor card 13 ₁ contains information tothe effect that the tap coefficient is generated from the set of 9 tapcoefficient seed data. The card controller 98 (see FIG. 6) in the videoprocessor card 13 ₁ controls the coefficient generator 96 in accordancewith the process information, thereby generating the set of tapcoefficients for the 64 classes for resolution enhancement/noise removalfrom the set of 9 tap coefficient seed data for resolutionenhancement/noise removal.

[0310] If the video processor cards 13 ₁ and 13 ₂ are loaded in thevideo processing interface 40, the video processor card 13 ₁ generatesthe set of tap coefficients for 64 classes for resolution enhancementfrom the set of 25 tap coefficient seed data for resolution enhancement,out of the set of 9 tap coefficient seed data for resolutionenhancement/noise removal and the set of 25 tap coefficient seed datafor resolution enhancement as shown in FIG. 18.

[0311] Referring to FIG. 18, the video processor card 13 ₂ generates theset of tap coefficient for 64 classes for noise removal from the set of25 tap coefficient seed data for noise removal.

[0312] Using the set of tap coefficients for 64 classes for resolutionenhancement, the video processor card 13 ₁ performs the data conversionprocess on first video data supplied from the frame memory 35 (see FIG.3), thereby outputting second video data with the resolution thereofimproved.

[0313] The second video data is fed to the video processor card 13 ₂ asthe first video data through the card interface 83 in the videoprocessing interface 40.

[0314] Using the set of tap coefficients for 64 classes for noiseremoval, the video processor card 13 ₂ perform the data conversionprocess on the first video data supplied thereto, thereby outputtingsecond video data with noise thereof removed.

[0315] The video data stored in the frame memory 35 is data converted byeach of the video processor card 13 ₁ and 13 ₂, thereby becoming videodata with the resolution thereof enhanced and noise removed.

[0316] The process information stored in the tap generation storage unit97 (see FIG. 6) in the video processor card 13 ₂ contains information tothe effect that the video processor card 13 ₁ generates the tapcoefficient from the set of 25 tap coefficient seed data for resolutionenhancement. The card controller 98 (see FIG. 6) in the video processorcard 13 ₁ controls the coefficient generator 96 in accordance with theprocess information of the video processor card 13 ₂ (in this case, theprocess information of the video processor card 13 ₂ is the maximum IDprocess information, and as illustrated in FIG. 8, the video processorcard 13 ₁ generates the tap coefficient in accordance with the maximumID process information). The video processor card 13 ₁ thus generatesthe set of tap coefficients for 64 classes for resolution enhancementfrom the set of 25 tap coefficient seed data for resolution enhancement.

[0317] Regardless of whether the video processor card 13 ₁ only or bothvideo processor cards 13 ₁ and 13 ₂ are loaded, the finally obtainedvideo data is enhanced in resolution and with noise removed.

[0318] However, when the video processor card 13 ₁ only is loaded, thevideo processor card 13 ₁ performs the data conversion process using theset of tap coefficient for the 64 classes for resolutionenhancement/noise removal serving dual purposes of enhancing resolutionand removing noise. When both the video processor card 13 ₁ and 13 ₂ areloaded, the video processor card 13 ₁ performs the data conversionprocess using the set of tap coefficient for the 64 classes forresolution enhancement serving the purpose of resolution enhancementonly, and furthermore, the video processor card 13 ₂ performs the dataconversion process using the set of tap coefficients for the 64 classesfor noise removal serving the purpose of noise removal only.

[0319] Even when the video processor card 13 ₁ only is loaded, theobtained video data is enhanced in resolution and with noise removed.When both the video processor card 13 ₁ and 13 ₂ are loaded, theresulting video data is enhanced more in resolution and with more noiseremoved.

[0320] The main unit 1 (see FIG. 1) of the television receiver becomeseven more sophisticated in performance by adding the video processorcard 13 ₂ subsequent to the loading of the video processor card 13 ₁.

[0321] The user thus has a motivation to purchase the video processorcard 13 ₂ even after he or she has purchased the video processor card 13₁.

[0322] When loaded, the video processor card 13 ₁ generates the set oftap coefficients for the 64 classes for resolution enhancement/noiseremoval from the set of 9 tap coefficient seed data for resolutionenhancement/noise removal. When both video processor cards 13 ₁ and 13 ₂are loaded, the video processor card 13 ₁ generates the set of tapcoefficients for the 64 classes for resolution enhancement from the setof 25 tap coefficient seed data for resolution enhancement. The numberof predictive taps is 9 when the video processor card 13 ₁ only isinstalled, while the number of predictive taps is 25 when both videoprocessor cards 13 ₁ and 13 ₂ are loaded. Because of the increasednumber predictive taps, the level of resolution enhancement with bothvideo processor cards 13 ₁ and 13 ₂ loaded is typically higher than thelevel of resolution enhancement with the video processor card 13 ₁ onlyloaded.

[0323] The main unit 1 of the television receiver enjoys furtherperformance improvements with resolution enhancement and noise removalwhen the video processor card 13 ₂ is added in addition to the alreadyloaded video processor card 13 ₁. But the type of performanceimprovements of the main unit 1 remains unchanged. As for the functionof the video processor card 13 ₁, the video processor card 13 ₁ changesfrom the function of enhancing resolution and removing noise to thefunction of enhancing resolution only. The function itself of the videoprocessor card 13 ₁ thus changes.

[0324] The video processor card 13 ₃ stores, as the tap generationinformation, the coefficient seed data for generating a tap coefficientfor expansion for expanding an image. The video processor card 13 ₁stores three types of coefficient seed data including an expansion/noiseremoval/resolution enhancement tap coefficient for expansion, noiseremoval, and resolution enhancement, and an expansion/resolutionenhancement tap coefficient for image expansion and resolutionenhancement.

[0325] When the video processor card 13 ₁ only is loaded, the videoprocessor card 13 ₁ performs the data conversion process using theexpansion/noise removal/resolution enhancement tap coefficients. Whenthe video processor card 13 ₂ is further loaded, the video processorcard 13 ₁ performs the data conversion process using theexpansion/resolution enhancement tap coefficients, and the videoprocessor card 13 ₂ performs the data conversion process using a noiseremoval tap coefficient. When the video processor cards 13 ₁ through 13₃ are loaded, the video processor cards 13 ₁ through 13 ₃ respectivelyperform the data conversion process using the resolution enhancement tapcoefficient, the data conversion process using noise removal tapcoefficient, and the data conversion process using expansion tapcoefficient.

[0326] In the above discussion, the video processor card 13 ₂ stores theset of 25 tap coefficient seed data for noise removal. Referring to FIG.19, the video processor card 13 ₂ may store a set of difference databetween the set of 25 tap coefficient seed data for noise removal andthe set of 25 tap coefficient seed data for resolution enhancementstored in the video processor card 13 ₁.

[0327] When both the video processor cards 13 ₁ and 13 ₂ are mounted inthe video processing interface 40, the card controller 98 (see FIG. 6)in the video processor card 13 ₂ reads the 25 tap coefficient seed datafor resolution enhancement from the video processor card 13 ₁ throughthe card interface 83 (see FIG. 5) in the video processing interface 40,and adds the difference data stored in the video processor card 13 ₂ tothe read 25 tap coefficient seed data for resolution enhancement. Thecard controller 98 in the video processor card 13 ₂ thus generates the25 tap coefficient seed data for noise removal. The card controller 98feeds the 25 tap coefficient seed data for noise removal to thecoefficient generator 96 (see FIG. 6), thereby generating the tapcoefficients for the 64 classes for noise removal.

[0328] The tap generation information stored in the video processor card13 ₂ is such difference data. The method of reconstructing thecoefficient seed data from the difference data is described in theprocess information of the video processor card 13 ₂. The cardcontroller 98 in the video processor card 13 ₂ references the processinformation, thereby reconstructing the coefficient seed data from thedifference data as the tap generation information.

[0329] The set of tap coefficients for the 64 classes for noise removal,used by the video processor card 13 ₂ for the data conversion process,is not generated without the 25 tap coefficient seed data for resolutionenhancement stored in the video processor card 13 ₁. A user, who has novideo processor card 13 ₁, is prevented from illegally obtaining the setof tap coefficients for the 64 classes for noise removal for use in thedata conversion process in the video processor card 13 ₂.

[0330] Another video processor card 13 _(i) may store difference databetween coefficient seed data for generating tap coefficients used bythe video processor card 13 _(i) in the data conversion process thereofand coefficient seed data for generating tap coefficients used by thevideo processor card 13 _(i-1), which is one level higher in card ID, inthe data conversion process thereof.

[0331]FIG. 20 illustrates another construction of the video processorcard 13 _(i). As shown, components identical to those described withreference to FIG. 6 are designated with the same reference numerals, andthe discussion thereof is omitted here. The video processor card 13 _(i)is basically identical in construction to the video processor card 13shown in FIG. 6 except that the video processor card 13 _(i) includes ashared memory space controller 100 substituted for the block of thecoefficient memory 94 with the coefficient memory 94 connected to theshared memory space controller 100.

[0332] The shared memory space controller 100 controls memory spaceshared by the coefficient memory 94 of the video processor card 13 _(i)and the coefficient memory 94 of another video processor card 13 _(i′).

[0333] If the coefficient memory 94 only of the video processor card 13_(i) is not sufficient in capacity, the shared memory space controller100 communicates with a shared memory space controller 100 of anothervideo processor card 13 _(i′) through the card interface 83 (see FIG. 5)of the video processing interface 40. If the coefficient memory 94 ofthe other video processor card 13 _(i′) has remaining capacity, theremaining space and the real memory space of the coefficient memory 94of the video processor card 13 _(i) are used as a virtual memory spaceof the video processor card 13 _(i).

[0334] The shared memory space controller 100 stores the set of tapcoefficients output from the coefficient normalizer 99 in the virtualmemory space (the real memory space of the coefficient memory 94 in thevideo processor card 13 _(i) or the real memory space of the coefficientmemory 94 in the other video processor card 13 _(i′)). Upon receivingthe class code from the class classifier 93, the shared memory spacecontroller 100 reads the set of tap coefficients of a classcorresponding to the class code from the virtual memory space, and feedsthe set of the tap coefficients to the predictor 95.

[0335] The shared memory space controller 100 looks equivalent to thecoefficient memory 94 shown in FIG. 6, if viewed from the coefficientnormalizer 99 and class classifier 93.

[0336] The required size of the virtual memory space depends on the sizeof the tap coefficients which is stored in the virtual memory space. Thesize of the tap coefficients to be stored in the virtual memory space iscalculated from the number of predictive taps N and the number ofclasses Z, if the size of on tap coefficient is known. The cardcontroller 98 references the number of predictive taps N and the numberof classes Z described in the process information in the tap generationstorage unit 97, thereby calculating an overall size of the tapcoefficients generated by the coefficient generator 96. The cardcontroller 98 feeds the overall size information as a required memorycapacity to the shared memory space controller 100.

[0337] As described above, the shared memory space controller 100reserves the virtual memory space having a required memory capacity fedfrom the card controller 98.

[0338] If the coefficient memory 94 of the video processor card 13 _(i)has remaining capacity, the shared memory space controller 100 permitsthe video processor card 13 _(i′) to use that remaining capacity inresponse to a request from a shared memory space controller 100 of theother video processor card 13 _(i′). The shared memory space controller100 sets and uses the memory of the video processor card 13 _(i) as thevirtual memory space.

[0339] The coefficient memories 94 are shared among a plurality of videoprocessor cards 13 for the following reason.

[0340] In the embodiment illustrated in FIG. 6, the capacity of thecoefficient memory 94 has not been discussed. It is simply assumed thatthe coefficient memory 94 has memory capacity large enough to store thetap coefficients output from the coefficient normalizer 99.

[0341] In the embodiment illustrated in FIG. 17, the video processorcard 13 ₁ generates the set of tap coefficients for the 64 classes forresolution enhancement/noise removal from the set of 9 tap coefficientseed data for resolution enhancement/noise removal. To store a total oftap coefficients for 64 classes with 9 taps per class, the videoprocessor card 13 ₁ needs a memory capacity of 64×9×S, where S is thesize of a single tap coefficient.

[0342] In the embodiment illustrated in FIG. 18, the video processorcard 13 ₁ generates the set of tap coefficients for the 64 classes forresolution enhancement from the set of 25 tap coefficient seed data forresolution enhancement. To store a total of tap coefficients for 64classes with 25 taps per class, the video processor card 13 ₁ needs amemory capacity of 64×25×S.

[0343] In the embodiment illustrated in FIG. 6, the coefficient memory94 in the video processor card 13 ₁ needs a memory size of 64×25×S.

[0344] The whole memory size of the coefficient memory 94, namely64×25×S, is used only when both the video processor cards 13 ₁ and 13 ₂are mounted. When the video processor card 13 ₁ only is mounted, only amemory of 64×9×S is used, and two-thirds of the coefficient memory 94(approximately equal to (25−9)/25) remains unused.

[0345] The coefficient memory 94 in the embodiment illustrated in FIG.20 has a memory capacity enough to store the set of the tap coefficientsof a predetermined number of classes Z by the predetermined number oftaps N with no unused memory capacity left.

[0346] The video processor card 13 ₁ and 13 ₂ are discussed again. Thevideo processor card 13 ₁ stores, as the tap generation information, theset of coefficient seed data for resolution enhancement/noise removaland the set of coefficient seed data for resolution enhancement as shownin FIG. 21. If the video processor card 13 ₁ only is mounted on thevideo processing interface 40, the set of tap coefficients for the 64classes for resolution enhancement/noise removal are generated from theset of coefficient seed data for resolution enhancement/noise removal asshown in FIG. 21. If both the video processor cards 13 ₁ and 13 ₂ aremounted on the video processing interface 40, the set of tapcoefficients for 48 classes for resolution enhancement is generated.

[0347] The video processor card 13 ₂ stores, as the tap generationinformation, the set of coefficient seed data for noise removal as shownin FIG. 22. When both video processor card 13 ₁ and 13 ₂ are mounted onthe video processing interface 40, the video processor card 13 ₂generates set of tap coefficients for 80 classes for noise removal.

[0348] For simplicity of explanation, the number of predictive taps,namely, the number of tap coefficients per class is constant value N,and is disregarded.

[0349] The coefficient memory 94 in each of the video processor cards 13₁ and 13 ₂ has a memory capacity for a set of tap coefficients for 64classes.

[0350] If the video processor card 13 ₁ only is mounted on the videoprocessing interface 40, the coefficient memory 94 in the videoprocessor card 13 ₁ stores the set of tap coefficients for 64 classesfor resolution enhancement/noise removal generated from the set ofcoefficient seed data for resolution enhancement/noise removal as shownin FIG. 23. In this case, the memory capacity of the coefficient memory94 in the video processor card 13 ₁ equals the size of the set of tapcoefficients for the generated 64 classes, and thus no memory capacityis left unused.

[0351] If both the video processor cards 13 ₁ and 13 ₂ are loaded in thevideo processing interface 40, the coefficient memory 94 in the videoprocessor card 13 ₁ stores a set of tap coefficients for 48 classes forresolution enhancement generated from a set of coefficient seed data forresolution enhancement as shown in FIG. 24. The size of the set of tapcoefficients for 48 classes is smaller than the memory capacity of thecoefficient memory 94 in the video processor card 13 ₁ by a set of tapcoefficients for 16 classes. A portion of the memory capacity of thecoefficient memory 94 equal to the set of tap coefficients for the 16classes remains unused.

[0352] However, if both the video processor cards 13 ₁ and 13 ₂ areloaded in the video processing interface 40, the video processor card 13₂ generates a set of tap coefficients for 80 classes for resolutionenhancement from a set of coefficient seed data for resolutionenhancement as shown in FIG. 25. The set of tap coefficients for 80classes cannot be wholly stored in the coefficient memory 94 which has amemory capacity for the storage of the set of tap coefficients for 64classes only. In the video processor card 13 ₂, the size of the set oftap coefficients for the 80 classes is larger than the memory capacityof the coefficient memory 94 by the set of tap coefficients for 16classes. To store the set of all tap coefficients, the coefficientmemory 94 in the video processor card 13 ₂ lacks a memory capacity equalto a size of the set of tap coefficients for 16 classes.

[0353] In the embodiment illustrated in FIG. 20, the shared memory spacecontroller 100 in the video processor card 13 ₂ requests a memory spaceR₁′ having a capacity for the set of tap coefficients for the 16 classesof the real memory space R₁ of the coefficient memory 94 from the sharedmemory space controller 100 in the video processor card 13 ₁.

[0354] The shared memory space controller 100 in the video processorcard 13 ₁ recognizes that the memory space required for itself is thesize for storing the tap coefficients for the 48 classes, and alsorecognizes that a capacity for the set of tap coefficients for the 16classes, out of the real memory space R₁ of the coefficient memory 94,remains unused. Referring to FIG. 26, the shared memory space controller100 in the video processor card 13 ₁ reserves a capacity to store thetap coefficients for the 48 classes as a virtual memory space r₁ of itsown. Furthermore, the shared memory space controller 100 in the videoprocessor card 13 ₁ permits the shared memory space controller 100 inthe video processor card 13 ₂ to use the memory space R₁′ having acapacity to store the set of tap coefficients for the 16 classes.

[0355] Referring to FIG. 26, the shared memory space controller 100 inthe video processor card 13 ₂ reserves a virtual memory space r₂ havinga capacity to store the set of tap coefficients for the 80 classes,namely, a combination of the real memory space R₂ of the coefficientmemory 94 in the video processor card 13 ₂ and the real memory R₁′permitted to store the set of tap coefficients for the 16 classes in thecoefficient memory 94 in the video processor card 13 ₁. The sharedmemory space controller 100 in the video processor card 13 ₂ storesthere the set of tap coefficients for the 80 classes.

[0356] The process of the video processor card 13 illustrated in FIG. 20will now be discussed with reference to a flow diagram illustrated inFIG. 27.

[0357] In step S51, the card controller 98 in the video processor card13 illustrated in FIG. 20 determines whether the video processor card 13has received the process information (the maximum ID processinformation) transmitted from the interface controller 81 (see FIG. 5)in step S5 as already discussed with reference to FIG. 7.

[0358] If it is determined in step S51 that the maximum ID processinformation has not been received, the algorithm proceeds to step S56,skipping steps S52 through S55.

[0359] If it is determined in step S51 that the maximum ID processinformation has been received, the algorithm proceed to step S52. Thecard controller 98 references the maximum ID process information,thereby recognizing the content of the data conversion process whichneeds to be performed by the card controller 98 itself like in step S12illustrated in FIG. 8. In accordance with the result of recognition, thecard controller 98 controls the tap extractors 91 and 92, and classclassifier 93.

[0360] In step S53, the card controller 98 recognizes the size of theset of tap coefficients generated by the coefficient generator 96, byreferencing the process information (the maximum ID process information)which is determined as being received from the video processinginterface 40. The card controller 98 then notifies the shared memoryspace controller 100 of the size of the set of tap coefficientsgenerated by the coefficient generator 96. The maximum ID processinformation contains information about the number of classes Z of thetap coefficients generated from the coefficient seed data and the numberof taps N, as the tap generation information. Based on these pieces ofinformation, the card controller 98 recognizes the size of the tapcoefficients generated in the coefficient generator 96, and feeds thisinformation to the shared memory space controller 100.

[0361] Also in step S53, the shared memory space controller 100 reservesthe virtual memory space having the size notified by the card controller98 as already discussed, and the algorithm then proceeds to step S54.

[0362] In step S54, the card controller 98 controls the coefficientgenerator 96 in accordance with the maximum ID process information likein step S13 illustrated in FIG. 8, thereby generating the tapcoefficients for each class from the tap generation information. The tapcoefficients are fed to the coefficient normalizer 99.

[0363] In step S55, the coefficient normalizer 99 normalizes the tapcoefficients from the coefficient generator 96, and feeds the tapcoefficients for each class to the shared memory space controller 100subsequent to the normalization. Also in step S55, the shared memoryspace controller 100 stores the tap coefficients for each class from thecoefficient normalizer 99 in the virtual memory space which is madeavailable in step S53. The algorithm then proceeds to step S56.

[0364] In step S56, the tap extractors 91 and 92 construct thepredictive tap and class tap with respect to a target pixel,respectively, in the same manner as in step S15 illustrated in FIG. 8.The predictive tap is fed from the tap extractor 91 to the predictor 95,and the class tap is fed from the tap extractor 92 to the classclassifier 93.

[0365] In step S57, the class classifier 93 classifies the target pixelbased on the class tap of the target pixel supplied from the tapextractor 92, and feeds the class code representing the class of thetarget pixel to the shared memory space controller 100. The algorithmproceeds to step S58.

[0366] In step S58, the shared memory space controller 100 reads andacquires, from the virtual memory space, the tap coefficient stored atan address corresponding to the class code supplied from the classclassifier 93. The tap coefficient is fed to the predictor 95 from theshared memory space controller 100. The algorithm then proceeds to stepS59.

[0367] In step S59, the predictor 95 performs prediction calculation ofequation (1) using the predictive tap output from the tap extractor 91and the tap coefficient supplied from the shared memory space controller100. The predictor 95 determines the pixel value of the target pixel,and feeds the pixel value to the card interface 83 (see FIG. 5). Thealgorithm loops to step S51 to repeat the same process.

[0368] A plurality of video processor cards 13 share a portion of thecoefficient memory 94, and reserves the virtual memory space of minimumsize capable of storing the tap coefficients. The memory capacity of thecoefficient memory 94 is thus conserved with no memory space leftunused.

[0369] Referring to FIGS. 21 through 26, the video processor card 13 ₂stores the set of tap coefficients for the 80 classes generated by thecoefficient generator 96 only when the video processor card 13 ₁ is alsoloaded together with the video processor card 13 ₂ on the videoprocessing interface 40. The data conversion process cannot be performedwith the video processor card 13 ₂ only. A user, who has no videoprocessor card 13 ₁, is prevented from performing illegally the dataconversion process with the video processor card 13 ₂ only.

[0370] If the spatial resolution of the image is enhanced to improveimage quality, the video processor card 13 ₁ stores, as the tapgeneration information, a set of 25 tap coefficient seed data asillustrated in FIG. 28. A learning process is performed with super-highresolution video data set as training data, and low-resolution data setas learning data. The low-resolution video data is obtained by degradingthe super-high resolution data into high-resolution video data with thespatial resolution lowered, and further by degrading the high-resolutionvideo data to the low-resolution video data with the spatial resolutionlowered. A set of tap coefficients for 64 classes for super-highresolution is thus generated, and then compressed, and the set of 25 tapcoefficient seed data for super-high resolution is then generated. Theset of 25 tap coefficient seed data is stored in the video processorcard 13 ₁ as the tap generation information.

[0371] As shown in FIG. 29, the video processor card 13 ₁ also stores,as the tap generation information, a set of 25 coefficient seed data inaddition to the set of 25 tap coefficient seed data for super-highresolution as illustrated in FIG. 28. A learning process is performedwith high-resolution video data set as training data, and low-resolutiondata set as learning data. The low-resolution video data is obtained bydegrading the high-resolution video data to the low-resolution videodata with the spatial resolution lowered. A set of tap coefficients for64 classes for enhancing the resolution to standard resolution is thusgenerated. The set of tap coefficients for the 64 classes is thencompressed, and the set of 25 tap coefficient seed data for standardresolution is then generated. The set of 25 tap coefficient seed data isstored in the video processor card 13 ₁ as the tap generationinformation.

[0372] If the video processor card 13 ₁ only is loaded in the videoprocessing interface 40, the tap coefficients for 64 classes forstandard resolution is generated from the 25 tap coefficient seed datafor standard resolution. Using the tap coefficients, the video processorcard 13 ₁ performs the data conversion process. If both the videoprocessor cards 13 ₁ and 13 ₂ are loaded in the video processinginterface 40, the tap coefficients for 64 classes for super-highresolution is generated from the 25 tap coefficient seed data forsuper-high resolution. Using the tap coefficients, the video processorcard 13 ₁ performs the data conversion process.

[0373] Regardless of whether the video processor card 13 ₁ only or boththe video processor cards 13 ₁ and 13 ₂ are loaded, the data conversionprocess of the same type of image quality improvement, namely, forresolution enhancement, is performed. However, if the video processorcard 13 ₁ only is loaded, the tap coefficients for 64 classes forresolution enhancement to standard resolution are used. If both thevideo processor cards 13 ₁ and 13 ₂ are loaded, the tap coefficients for64 classes for resolution enhancement to super-high resolution,different from the tap coefficients for 64 classes for resolutionenhancement to standard resolution, are used. The video data resultingfrom the data conversion process is improved more in resolution.

[0374] In this embodiment, the generation of the tap coefficient w_(n)from the coefficient seed data β_(m,n) is carried out in accordance withthe tap calculation equation (9).

[0375] When the spatial resolution of the image is enhanced to improveimage quality, the video processor card 13 ₁ stores, as the tapgeneration information, a set of 25 tap coefficient seed data in thesame way already discussed with reference to FIGS. 28 and 29. A learningprocess is performed with high resolution video data set as trainingdata, and low-resolution data, obtained by degrading the spatialresolution of the high-resolution video data and set as learning data asshown in FIG. 30. A set of tap coefficients for 64 classes forsuper-high resolution is thus generated, and then compressed, and theset of 25 tap coefficient seed data for super-high resolution is thengenerated. The set of 25 tap coefficient seed data is stored in thevideo processor card 13 ₁ as the tap generation information. The tapcalculation equation (9) for compressing the tap coefficients into thecoefficient seed data is M′ term equation having the number of terms M′.A set of coefficient seed data for one tap includes M′ pieces ofcoefficient seed data.

[0376] The video processor card 13 ₁ stores, as the tap generationinformation, a set of coefficient seed data in addition to the set of 25tap coefficient seed data for resolution enhancement shown in FIG. 30. Alearning process is performed with high resolution video data set astraining data, and low-resolution data, obtained by degrading thespatial resolution of the high-resolution video data and set as learningdata as shown in FIG. 31. A set of tap coefficients for 64 classes forresolution enhancement is thus generated, and then compressed. The setof 25 tap coefficients for resolution enhancement is generated. The setof 25 tap coefficient seed data is stored in the video processor card 13₁ as the tap generation information. The tap calculation equation (9)for compressing the tap coefficients into the coefficient seed data isM″ term equation having the number of terms M″, which is greater thanM′. A set of coefficient seed data for one tap includes M″ pieces ofcoefficient seed data.

[0377] If the video processor card 13 ₁ only is loaded in the videoprocessing interface 40, the tap coefficients for 64 classes forresolution enhancement are generated from the 25 tap coefficient seeddata for resolution enhancement with M′ pieces of coefficient seed dataper tap. The video processor card 13 ₁ performs the data conversionprocess using the tap coefficients. If both the video processor card 13₁ and 13 ₂ are loaded in the video processing interface 40, the tapcoefficients for 64 classes for resolution enhancement are generatedfrom the 25 tap coefficient seed data for resolution enhancement with M″pieces of coefficient seed data per tap. The video processor card 13 ₁performs the data conversion process using the tap coefficients.

[0378] Regardless of whether the video processor card 13 ₁ only or boththe video processor cards 13 ₁ and 13 ₂ are loaded, the data conversionprocess of the same type of image quality improvement, namely, forresolution enhancement, is performed. If the video processor card 13 ₁only is loaded, the tap coefficients for use in the data conversionprocess are generated from the 25 tap coefficient seed data forresolution enhancement with M′ pieces of coefficient seed data per tapin accordance with the tap calculation equation (9) having the number ofterms M′. If both video processor cards 13 ₁ and 13 ₂ are loaded, thetap coefficients for use in the data conversion process are generatedfrom the 25 tap coefficient seed data for resolution enhancement with M″pieces of coefficient seed data per tap in accordance with the tapcalculation equation (9) having the number of terms M″, which is largerthan M′. The tap coefficients are reconstructed with a higher accuracy.Like in the cases illustrated in FIGS. 28 and 29, the video processorcard 13 ₁ improves more the spatial resolution of the video data throughthe data conversion process when both video processor cards 13 ₁ and 13₂ are loaded than when the video processor card 13 ₁ only is loaded.

[0379]FIG. 32 illustrates a second construction of the video processinginterface 40 of FIG. 3. As shown, components identical to thosediscussed with reference to FIG. 5 are designated with the samereference numerals and the discussion thereof is omitted as appropriate.

[0380] The video processing interface 40 in the embodiment illustratedin FIG. 32 reads the tap generation information from the video processorcard 13 to be discussed with reference to FIG. 33 and mounted therein,and generates the tap coefficients. Using the tap coefficients, thevideo processing interface 40 performs the data conversion process onthe video data stored in the frame memory 35 (see FIG. 3).

[0381] Tap extractors 131 and 132 are supplied with first video datawhich is read from the frame memory 35 through the memory interface 82and which is to be data converted.

[0382] Like the tap extractor 91 shown in FIG. 6, the tap extractor 131successively sets each pixel constituting second video data as a targetpixel, and extracts, as predictive taps, several (pixel values of)pixels constituting the first video data which is used to predict thepixel value of the target pixel.

[0383] Like the tap extractor 92 shown in FIG. 6, the tap extractor 132extracts, as a class tap, several pixels constituting the first videodata which is used to classify the target pixel in one of a plurality ofclasses.

[0384] The tap extractors 131 and 132 are supplied with a control signalfrom the interface controller 81. The tap structure of the predictivetap constructed by the tap extractor 131 and the tap structure of theclass tap constructed by the tap extractor 132 are set in accordancewith the control signal from the interface controller 81. Specifically,the interface controller 81 reads, through the card interface 83, theprocess information from the video processor card 13 to be discussedwith reference to FIG. 33 loaded in the video processing interface 40.Based on the process information, the interface controller 81 controlsthe tap structure in each of the tap extractors 131 and 132.

[0385] The predictive tap obtained in the tap extractor 131 is fed to apredictor 135, and the class tap obtained in the tap extractor 132 isfed to the interface controller 81.

[0386] Based on information supplied from the interface controller 81,the class classifier 133 classifies the target pixel, and feeds a classcode of the resulting class to a coefficient memory 134. The interfacecontroller 81 feeds, through the card interface 83, the class tap of thetarget pixel supplied from the tap extractor 132 to the video processorcard 13 to be discussed with reference to FIG. 33 and loaded in thevideo processing interface 40. The interface controller 81 requests thevideo processor card 13 to classify the target pixel. The interfacecontroller 81 receives, through the card interface 83, the class codewhich the video processor card 13 transmits in response to the requestsubsequent to the class classification. The class code is then fed tothe class classifier 133. The class classifier 133 synthesizes classcodes, supplied from the interface controller 81, as a result of atleast one class classifications performed by the video processor card 13loaded in the video processing interface 40. The class classifier 133results in a class code as a final class classification result, andfeeds the class code to the coefficient memory 134.

[0387] For example, the class classification method of the videoprocessor card 13 is designated based on the process information whichthe interface controller 81 reads from the video processor card 13loaded in the video processing interface 40.

[0388] The coefficient memory 134 stores the tap coefficients for eachclass supplied from the coefficient generator 136. The coefficientmemory 134 supplies the predictor 135 with tap coefficients, out of thestored tap coefficients, stored as an address corresponding to the classcode supplied from the class classifier 133 (the tap coefficients of aclass represented by the class code supplied from the class classifier133).

[0389] Like the predictor 95 illustrated in FIG. 6, the predictor 135acquires the predictive tap output from the tap extractor 131 and thetap coefficient output from the coefficient memory 134, and performsprediction calculation defined by equation (1) for determining apredictive value of the true value of the target pixel, using thepredictive tap and the tap coefficient. The predictor 135 determines andoutputs (the predictive value of) the pixel value of the target pixel,namely, the pixel value of the pixel constituting the second video data.

[0390] The coefficient generator 136 receives the tap generationinformation which the interface controller 81 reads, through the cardinterface 83, from the video processor card 13 loaded in the videoprocessing interface 40. Based on the tap generation information, thecoefficient generator 136 generates the tap coefficients for each class.The tap coefficients for each class are fed to the coefficient memory134 for storage.

[0391]FIG. 33 illustrates the construction of the video processor card13 wherein the video processing interface 40 is constructed asillustrated in FIG. 32. As shown, components identical to thosedescribed with reference to FIG. 6 are designated with the samereference numerals, and the discussion thereof is omitted asappropriate. The video processor card 13 shown in FIG. 33 includes theclass classifier 93, tap generation storage unit 97, and card controller98 shown in FIG. 6.

[0392] The operation of the video processing interface 40 shown in FIG.32 is discussed below with reference to a flow diagram illustrated inFIG. 34.

[0393] The video processing interface 40 performs process steps in stepsS71 through S74, which are the same process steps as in steps S1 throughS4 as illustrated in FIG. 7.

[0394] In step S74, the interface controller 81 determines a videoprocessor card 13 _(i) to be an effective card. In step S75, theinterface controller 81 reads the tap generation information from allvideo processor cards 13 _(i) as effective cards through the cardinterface 83. Specifically, the interface controller 81 requests the tapgeneration information through the card interface 83 from the cardcontroller 98 (see FIG. 33) in the video processor card 13 _(i) as aneffective card. The card controller 98 reads the tap generationinformation from the tap generation storage unit 97 and feeds the tapgeneration information to the interface controller 81 through the cardinterface 83. In this way, the interface controller 81 acquires all tapgeneration information from the video processor card 13 _(i) as aneffective card.

[0395] For example, if the video processor cards 13 ₁ through 13 ₃ areloaded in the video processing interface 40, all video processor cards13 ₁ through 13 ₃ become effective cards. The interface controller 81thus receives the tap generation information from all video processorcards 13 ₁ through 13 ₃. If the video processor cards 13 ₁, 13 ₂, and 13₄ are loaded in the video processing interface 40, the video processorcards 13 ₁ and 13 ₂ become effective cards. The interface controller 81receives the tap generation information from each of the video processorcards 13 ₁ and 13 ₂.

[0396] In step S76, the interface controller 81 recognizes the contentof the data conversion process to be performed, based on the processinformation read from the video processor card 13 _(i(max)) having themaximum card ID (the maximum ID process information) out of the processinformation read from each video processor card 13 _(i) as an effectivecard. Based on the recognition result, the interface controller 81controls the tap extractors 131 and 132, class classifier 133, and videoprocessor card 13 _(i) as the effective card.

[0397] The interface controller 81 sets the operation mode of the tapextractor 131 or 132 to construct the tap structure of the predictivetap or the class tap described in the maximum ID process information.

[0398] The interface controller 81 sets the operation mode of the classclassifier 133 to synthesize the class code of an effective card basedon a synthesis method described in the maximum ID process information.Specifically, the process information of the video processor card 13_(i) contains a description of the synthesis method of the class code(for example, information for generating a final class code by attachinga class code, obtained in the video processor card 13 ₂, as the leastsignificant bits to a class code obtained in the video processor card 13₁). The interface controller 81 sets the operation mode of the classclassifier 133 to synthesize the class codes in accordance with thatinformation.

[0399] The interface controller 81 sets the operation mode of each videoprocessor card 13 _(i) as an effective card so that a classclassification is performed in accordance with the class classificationmethod described in the maximum ID process information. Specifically,the process information of the video processor card 13 _(i) contains adescription of the class classification method of the classclassification to be carried out by the class classifier 93 in each ofthe video processor cards 13 ₁ through 13 _(i). The interface controller81 sets the operation mode of the class classifier 93 (see FIG. 33) ofthe video processor card 13 _(i) as an effective card.

[0400] In step S77, the interface controller 81 feeds the tap generationinformation read from the video processor card 13 _(i) as an effectivecard to the coefficient generator 136, thereby generating the tapcoefficients for each class from the tap generation information.

[0401] When the coefficient generator 136 generates the tap coefficientsfor each class, the algorithm proceeds to step S78. The coefficientgenerator 136 performs a normalization process on the tap coefficientsfor each class for level adjustment, and then feeds the normalized tapcoefficients to the coefficient memory 134 for storage.

[0402] In step S79, the tap extractor 131 sets the video data, stored inthe frame memory 35 (see FIG. 3) and supplied from the memory interface82, as first video data which is to be data converted. Each pixelconstituting second video data (the video data subsequent to the dataconversion process) corresponding to the first video data issuccessively set as a target pixel. The tap extractor 131 extracts thepixel of the first video data having the predictive tap with the tapstructure set in step S76 with respect to the target pixel. Also in stepS79, the tap extractor 132 extracts a pixel of the first video datahaving the class tap with the tap structure set in step S76 with respectto the target pixel. The predictive tap is fed from the tap extractor131 to the predictor 135, and the class tap is fed from the tapextractor 132 to the interface controller 81.

[0403] Upon receiving the class tap from the tap extractor 132, theinterface controller 81 controls the card interface 83 in step S80,thereby supplying the effective video processor card 13 _(i) with theclass tap from the tap extractor 132, and thereby requesting theeffective video processor card 13 _(i) to perform class classification.

[0404] As will be discussed later with reference to FIG. 35, the videoprocessor card 13 _(i), which is currently effective, classifies thetarget pixel based on the class tap from the interface controller 81 inresponse to the request from the interface controller 81. The class codeas a result of class classification is then fed to the interfacecontroller 81 through the card interface 83.

[0405] In step S81, the interface controller 81 determines whether theclass codes have been received from all currently effective videoprocessor cards 13 _(i) as a result of class classification.

[0406] If it is determined in step S81 that the class codes have notbeen received from all currently effective video processor cards 13 _(i)as a result of class classification, step S81 is repeated.

[0407] If it is determined in step S81 that the class codes have beenreceived from all currently effective video processor cards 13 _(i) as aresult of class classification, all class codes are fed to the classclassifier 133. The algorithm proceeds to step S82.

[0408] In step S82, the class classifier 133 synthesizes the class codessupplied from the effective video processor cards 13 _(i) through theinterface controller 81, thereby resulting in a class code as a resultof final class classification of the target pixel. The class code isthen fed to the coefficient memory 134. The algorithm proceeds to stepS83.

[0409] In step S83, the coefficient memory 134 reads and outputs the tapcoefficient stored at an address corresponding to the class codesupplied from the class classifier 133, namely, the tap coefficient ofthe class corresponding to the class code. In step S83, the predictor135 receives the tap coefficient output from the coefficient memory 134.The algorithm proceeds to step S84.

[0410] In step S84, the predictor 135 performs prediction calculationexpressed by equation (1) using the predictive tap output from the tapextractor 131, and the tap coefficient obtained from the coefficientmemory 134. In this way, the predictor 135 determines the pixel value ofthe target pixel, and then feeds the pixel value to the line-by-lineconverter 85. The algorithm loops to step S71 to repeat the aboveprocess.

[0411] The process of the video processor card 13 ₁, which becomes aneffective card, is discussed with reference to a flow diagramillustrated in FIG. 35.

[0412] In step S91, the card controller 98 in the effective videoprocessor card 13 _(i) determines whether the video processor card 13_(i) has received a class classification request together with the classtap from (the interface controller 81 of) the video processing interface40. If it is determined that the class classification request has notbeen received, step S91 is repeated.

[0413] If it is determined in step S91 that the video processor card 13_(i) has received the class classification request, the algorithmproceeds to step S92. The card controller 98 receives the classclassification request together with the class tap from the videoprocessing interface 40. The card controller 98 requests the classclassifier 93 to perform class classification in response to the classtap. The class classifier 93 classifies the target pixel in accordancewith the class tap using the class classification method set in step S76as illustrated in FIG. 34. The resulting class code is fed to the cardcontroller 98.

[0414] In step S93, the card controller 98 transmits the class code fromthe class classifier 93 to the video processing interface 40. Thealgorithm loops to step S91 to repeat the same process.

[0415] Discussed next are the class classification of the classclassifier 93 (see FIG. 33) of the effective video processor card 13_(i) and the process of the class classifier 133 (see FIG. 32) of thevideo processing interface 40 which synthesizes the class code as aresult of class classification.

[0416] For example, the tap extractor 132 constructs a class tap formedof nine pixels, namely 3×3 pixels, as target pixels.

[0417] It is now assumed that the video processor cards 13 ₁ through 13₃ are loaded in the video processing interface 40, and are set to beeffective. The class classifier 93 (see FIG. 33) in the video processorcard 13 _(i) classifies the class tap using 1-bit ADRC process, therebyoutputting a 9 bit class code.

[0418] The class classifier 93 (see FIG. 33) in the video processor card13 ₂ detects a motion vector of the pixel at the center of the class tap(center pixel), and compares the magnitude of the motion vector with apredetermined threshold. The class classifier 93 thus determines whetheror not the center pixel has moved, thereby outputting a 1 bit class coderepresenting the presence or absence of motion of the center pixel. Theclass classifier 93 in the video processor card 13 ₂ stores one frameold video data that is supplied as a class tap one frame earlier, andperforms block matching with the one frame old video data using theclass tap, thereby detecting the motion vector. The class classifier 93in the video processor card 13 ₂ determines that the center pixel hasmoved if the magnitude of the motion vector is above the predeterminedthreshold; otherwise, the class classifier 93 in the video processorcard 13 ₂ determines that the center pixel remains stationary.

[0419] The class classifier 93 (see FIG. 33) in the video processor card13 ₃ calculates a difference between the center pixel of the class tapand each pixel adjacent to the center pixel, and compares the absolutevalues of the differences with a predetermined threshold. The classclassifier 93 determines the presence or absence of an edge in thecenter pixel, thereby outputting a 1 bit class code representing thepresence or absence of the edge. For example, the class classifier 93determines that there is an edge if any of the absolute values of thedifferences of the pixels with respect to the center pixel is above thepredetermined threshold; otherwise, the class classifier 93 determinesthat there is no edge.

[0420] Hereinafter, the class classification performed by the classclassifier 93 in the video processor card 13 _(i) is referred to as ani-th class classification, and a class code obtained as a result of thei-th class classification is referred to as an i-th class code.

[0421] The class classifier 133 (see FIG. 32) in the video processinginterface 40 attaches a second class code obtained through a secondclass classification of the class classifier 93 in the video processorcard 13 ₂ as the least significant bit to a first class code of a firstclass classification of the class classifier 93 in the video processorcard 13 ₁, and further attaches a third class code obtained through athird class classification of the class classifier 93 in the videoprocessor card 13 ₃ as the least significant bit to the leastsignificant bit of the first class code.

[0422] If the video processor card 13 ₁ only is loaded in the videoprocessing interface 40, the class classifier 133 (see FIG. 32) in thevideo processing interface 40 outputs the first 9 bit class codeobtained through the first class classification of the class classifier93 of the video processor card 13 ₁ as a class code which is final(hereinafter referred to as a final class code).

[0423] If both video processor cards 13 ₁ and 13 ₂ are loaded in thevideo processing interface 40, the class classifier 133 (see FIG. 32)attaches the 1 bit second class code obtained through the second classclassification of the class classifier 93 in the video processor card 13₂ as the least significant bit to the 9 bit first class code obtainedthrough the first class classification of the class classifier 93 in thevideo processor card 13 ₁, and outputs the resulting 10 bit code as afinal class code.

[0424] If the video processor cards 13 ₁ through 13 ₃ are loaded in thevideo processing interface 40, the class classifier 133 (see FIG. 32) inthe video processing interface 40 attaches the 1 bit second class codeobtained through the second class classification of the class classifier93 in the video processor card 13 ₂ as the least significant bit to the9 bit first class code obtained through the first class classificationof the class classifier 93 in the video processor card 13 ₁, and furtherattaches a 1 bit third class code obtained through the third classclassification of the class classifier 93 in the video processor card 13₃, and outputs the resulting 11 bits as a final class code.

[0425] When the video processor card 13 ₁ through 13 _(i) are loaded inthe video processing interface 40, the class code output from the classclassifier 133 is incremented by 1 from the class code with the videoprocessor card 13 ₁ through 13 _(i−1) loaded, the class number changesdepending on the number of video processor cards 13 _(i) loaded in thevideo processing interface 40 as effective cards as illustrated in FIG.36.

[0426] If the video processor card 13 ₁ only is loaded in the videoprocessing interface 40, the final class code becomes 9 bits, and theclass number becomes 512 (=2⁹) (see the leftmost column in FIG. 36). Ifthe video processor card 13 ₁ and 13 ₂ are loaded in the videoprocessing interface 40, the final class code is 10 bits, and the numberof classes becomes 1024 (2¹⁰) classes (see the second column from theleft in FIG. 36). If the video processor card 13 ₁ through 13 ₃ areloaded in the video processing interface 40, the final class code is 11bits, and the number of classes becomes 2048 (2¹¹) classes (see therightmost column in FIG. 36).

[0427] The learning device illustrated in FIG. 11 performs a learningprocess using the class classification, which outputs the first classcode obtained through the first class classification, as the classclassification method of the class classifier 117. The learning devicethen generates a set of tap coefficients for 512 classes using the firstclass classification, and compresses the set of tap coefficients for the512 classes into a set of coefficient seed data according to the firstclass classification. The video processor card 13 ₁ stores the set ofcoefficient seed data according to the first class classification as thetap generation information.

[0428] The learning device illustrated in FIG. 11 performs a learningprocess using the class classification, which outputs a class code whichis obtained by attaching a second class code obtained using a secondclass classification as the least significant bit to the first classcode obtained using the first class classification, as the classclassification method of the class classifier 117. The learning devicegenerates a set of tap coefficients for 1024 classes according to thefirst and second class classifications, and compresses the set of tapcoefficients for the 1024 classes into a set of coefficient seed dataaccording to the first and second class classification. The videoprocessor card 13 ₂ thus stores the set of coefficient seed dataaccording to the first and second class classifications.

[0429] If the video processor card 13 ₁ only is loaded in the videoprocessing interface 40, the video processing interface 40 reads thecoefficient seed data according to the first classification stored asthe tap generation information in the highest order video processor card13 ₁ among effective cards as shown in FIG. 37. The coefficientgenerator 136 generates a set of tap coefficients for the 512 classesaccording to the first class classification from the coefficient seeddata. The class classifier 93 in the video processor card 13 ₁ performsthe first class classification, thereby outputting a 9 bit first classcode. The class classifier 133 in the video processing interface 40outputs the first class code as a final class code. The video processorcard 13 ₁ performs the data conversion process (the calculation definedby equation (1) and performed by the predictor 135 (see FIG. 32)) usingthe set of tap coefficients corresponding to the 9 bit first class code,among the set of tap coefficients for the 512 classes according to thefirst class classification.

[0430] If the video processor card 13 ₁ and 13 ₂ are loaded in the videoprocessing interface 40, the video processing interface 40 reads a setof coefficient seed data according to the first and second classclassification stored as the tap generation information in the lowestorder video processor card 13 ₂ of the effective cards as shown in FIG.28. The coefficient generator 136 generates, from the set of coefficientseed data, a set of tap coefficients for 1024 classes according to thefirst and second class classifications. The class classifier 93 in thevideo processor card 13 ₁ performs the first class classification,thereby outputting a 9 bit first class code, and the class classifier 93in the video processor card 13 ₂ performs the second classclassification, thereby outputting a 1 bit second class code. The classclassifier 133 in the video processing interface 40 attaches the secondclass code as the least significant bit to the first class code, therebygenerating and outputting a final 10 bit class code. Furthermore, thevideo processing interface 40 uses, in the data conversion process, theset of tap coefficients corresponding to the final 10 bit class code,out of the set of tap coefficients for the 1024 classes according to thefirst and second class classifications.

[0431] As the number of video processor cards 13 _(i) loaded in thevideo processing interface 40 and set as effective cards increases, thedata conversion process is performed using the tap coefficients with thelarger number of classes. The image quality of the video data is thusfurther improved.

[0432] For example, if the learning process for learning the set of tapcoefficients based on the first and second class classifications and thelearning process for learning the set of tap coefficients based on thefirst class classification are performed using the same education pair,the set of tap coefficients based on the first and second classclassification has the number of classes larger than the set of tapcoefficients based on the first class classification. The dataconversion process which is performed using the set of tap coefficientsbased on the first and second class classification results in theadvantage of image quality improvements over the data conversion processwhich is performed using the set of tap coefficients based on the firstclass classification.

[0433] The video processor cards 13 ₁ and 13 ₂ may store the coefficientseed data for performing the same image quality improvements. The videoprocessor card 13 ₁ and 13 ₂ may also store the coefficient seed datafor performing different image quality improvements. For example, thevideo processor card 13 ₁ stores the coefficient seed data forperforming noise removal, and the video processor card 13 ₂ stores thecoefficient seed data for performing resolution enhancement. The videoprocessing interface 40 illustrated in FIG. 32 performs the dataconversion process on the video data stored in the frame memory 35 (seeFIG. 3) using the tap coefficients generated from the coefficient seeddata stored in the video processor card 13 ₁, and further performs thedata conversion process on the resulting video data using the tapcoefficients generated from the coefficient seed data stored in thevideo processor card 13 ₂.

[0434] The video processor card 13 ₂ may store the coefficient seed datafor achieving another type of image quality improvements in addition tothe coefficient seed data for image quality improvements stored in thevideo processor card 13 ₁. Specifically, the video processor card 13 ₁stores the coefficient seed data for noise removal, and the videoprocessor card 13 ₂ stores the coefficient seed data for noise removaland resolution enhancement. In this embodiment, the number of classes oftap coefficients generated from the coefficient seed data of the videoprocessor card 13 ₂ is twice the number of classes of tap coefficientsgenerated from the coefficient seed data of the video processor card 13₁. The degree of noise removal achieved by the tap coefficientsgenerated from the coefficient seed data of the video processor card 13₂ remains almost the same as the degree of noise removal achieved by thecoefficient seed data generated from the coefficient seed data of thevideo processor card 13 ₁. When the video processor cards 13 ₁ and 13 ₂are loaded, noise removal is performed at the same degree as when thevideo processor card 13 ₁ only is loaded, and further the resolution isimproved.

[0435] The video processor card 13 _(i) other than the video processorcards 13 ₁ and 13 ₂ may store similar tap coefficients. As lower videoprocessor cards 13 _(i) are successively loaded in succession to thehighest order video processor card 13 ₁, the number of types of imagequality improvements increases. Since the functions of the main unit 1of the television receiver additionally increase, the user has amotivation to purchase the video processor card 13.

[0436] The video processor card 13 _(i) other than the highest ordervideo processor card 13 ₁ may store the difference data rather than thecoefficient seed data itself as already discussed with reference to FIG.19.

[0437] In the above discussion, the video processor card 13 _(i)illustrated in FIG. 33 stores, as the tap generation information, thecoefficient seed data or the data of difference between the coefficientseed data of the video processor card 13 _(i) and the coefficient seeddata of the video processor card 13 _(i-1), which is one order higher.The video processor card 13 _(i), except the highest order videoprocessor card 13 ₁, stores, as the tap generation information,information for generating new tap coefficients (information other thanthe difference data) from the tap coefficients used in the videoprocessor card 13 _(i-1) which is one order higher.

[0438] If the video processor card 13 ₁ and 13 ₂ are loaded in the videoprocessing interface 40, the coefficient generator 136 (see FIG. 32) inthe video processing interface 40 generates the set of tap coefficientsbased on the first class classification from the set of coefficient seeddata based on the first class classification stored in the videoprocessor card 13 ₁ as shown in FIG. 39. Furthermore, the coefficientgenerator 136 in the video processing interface 40 generates the set oftap coefficients based on the first and second class classification fromthe set of tap coefficients based on the first class classification andthe tap generation information stored in the video processor card 13 ₂.

[0439] Discussed next is the tap generation information for generatingthe set of tap coefficients, based on the first and second classclassification, together with the set of tap coefficients based on thefirst class classification.

[0440] The first class code from the first class classification may be afinal class code. The final class code is then referred to as a firstsynthesis class code. A final class code, which is obtained by attachinga second class code based on the second class classification as theleast significant bit to the first class code, is referred to as asecond synthesis class code. A final class code, which is obtained byattaching the second class code as the least significant bit to thefirst class code, and then attaching a third class code to the secondsynthesis class code, is referred to as a third synthesis class code.

[0441] In this case, the second synthesis class code is the firstsynthesis class code with the 1 bit second class code attached theretoas the least significant bit, and the third synthesis class code is thesecond synthesis class code with the 1 bit third class code attachedthereto.

[0442] A first synthesis class code #c may be correspondingly associatedwith two second synthesis class codes, namely, a second synthesis classcode #c0 which is obtained by attaching zero of 1 bit as the leastsignificant bit to the first synthesis class code #c and a secondsynthesis class code #c1 which is obtained by attaching 1 of 1 bit asthe least significant bit to the first synthesis class code #c.Similarly, a second synthesis class code #c′ may be correspondinglyassociated with two third synthesis class codes, namely, a thirdsynthesis class code #c′0 which is obtained by attaching zero of 1 bitas the least significant bit to the second synthesis class code #c′ anda third synthesis class code #c′1 which is obtained by attaching 1 of 1bit as the least significant bit to the second synthesis class code #c′.

[0443] There is one set of tap coefficients for one class code. If thefirst synthesis class code #c corresponds to the two second synthesisclass codes #c0 and #c1 as described above, a set of tap coefficients ofthe first synthesis class code #c based on the first classclassification corresponds to sets of tap coefficients of the secondsynthesis class codes #c0 and #c1 based on the first and second classclassifications as shown in FIG. 40.

[0444] The tap generation information may be used to generate the setsof tap coefficients of the second synthesis class codes #c0 and #c1based on the first and second class classifications together with theset of tap coefficients of the first synthesis class code #c based onthe first class classification.

[0445] With reference to a flow diagram illustrated in FIG. 41, themethod of generating the tap generation information is discussed. Theset of tap coefficients of the second synthesis class code #c0 based onthe first and second class classification is generated here.

[0446] In step S101, the center tap coefficient of the set of tapcoefficients of the first synthesis class code #c is compared with thecenter tap coefficient of the set of tap coefficients of the secondsynthesis class code #c0 correspondingly associated with the firstsynthesis class code #c.

[0447] The center tap coefficient of the set of tap coefficients at agiven class refers to a tap coefficient which is centered when the tapcoefficients are arranged in the order of tap numbers at the class.Referring to FIG. 40, for example, if the arrangement of the tapcoefficients of the first synthesis class code #c in the order of tapnumbers is (0.2, −0.1, 0.8, −0.1, 0.2), the center tap coefficient is0.8. If the arrangement of the tap coefficients of the second synthesisclass code #c0 in the order of tap numbers is (0.3, −0.4, 1.2, −0.4,0.3), the center tap coefficient is 1.2.

[0448] After comparing the center tap coefficients in step S101, thealgorithm proceeds to step S102. It is then determined in step S102whether the center tap coefficient of the second synthesis class code#c0 is (equal to or) larger than the center tap coefficient of the firstsynthesis class code #c.

[0449] If it is then determined in step S102 that the center tapcoefficient of the second synthesis class code #c0 is larger than thecenter tap coefficient of the first synthesis class code #c, thealgorithm proceeds to step S103. Determined in step S103 is the rate ofchange (ratio) of each positive tap coefficient of the set of tapcoefficients of the second synthesis class code #c0 with respect to thecorresponding tap coefficient of the first synthesis class code #c. Alsoin step S103, the mean value of the rate of change is determined, and isthen referred to as mean rate of change v.

[0450] In the example of tap coefficients shown in FIG. 40, the centertap coefficient of the second synthesis class code #c0 is 1.2 and islarger than the center tap coefficient of the first synthesis class code#c0 , which is 0.8. Determined in step S103 is the rate of change(ratio) of each of the positive tap coefficients of the tap coefficientsin the second synthesis class code #c0 , namely, the first tapcoefficient 0.3, the third tap coefficient (the center tap coefficient)1.2, and the fifth tap coefficient 0.3, with respect to thecorresponding tap coefficients of the first synthesis class code #c.

[0451] Since the first, third, and fifth tap coefficients of the firstsynthesis class code #c are respectively 0.2, 0.8, and 0.2, therespective rates of change of the first, third, and fifth tapcoefficients are 0.3/0.2, 1.2/0.8, and 0.3/0.2.

[0452] The means value of the rates of change v is 1.5.

[0453] The algorithm proceeds from step S103 to S104. A positive sign(+) is attached to the mean rate of change v, and the mean rate ofchange v becomes +v. The +v is set as the tap generation information(for the second synthesis class code #c0 ) for generating, from the setof tap coefficients of the first synthesis class code #c, the set of tapcoefficients of the second synthesis class code #c0 corresponding to theset of tap coefficients of the first synthesis class code #c. Theprocess ends.

[0454] In this example, +1.5 is the tap generation information forgenerating the set of tap coefficients of the second synthesis classcode #c0 from the set of tap coefficients of the first synthesis classcode #c.

[0455] If it is then determined in step S102 that the center tapcoefficient of the second synthesis class code #c0 is not larger thanthe center tap coefficient of the first synthesis class code #c, thealgorithm proceeds to step S105. Determined in step S105 is the rate ofchange (ratio) of each negative tap coefficient of the set of tapcoefficients of the second synthesis class code #c0 with respect to thecorresponding tap coefficient of the first synthesis class code #c. Alsoin step S105, the mean value of the rate of change is determined, and isthen referred to as mean rate of change v.

[0456] The algorithm proceeds from step S105 to S106. A negative sign(−) sign is attached to the mean rate of change v, and the mean rate ofchange v becomes −v. The −v is set as the tap generation information forgenerating, from the set of tap coefficients of the first synthesisclass code #c, the set of tap coefficients of the second synthesis classcode #c0 corresponding to the set of tap coefficients of the firstsynthesis class code #c. The process ends.

[0457] Generated next is the tap generation information for generatingthe set of tap coefficients of the second synthesis class code #c1 basedon the first and second class classifications as shown in FIG. 40. Theset of tap coefficients of the second synthesis class code #c1 is (0.17,−0.02, 0.7, −0.02, 0.17). The center tap coefficient is 0.7, which issmaller than the center tap coefficient of the tap coefficients of thefirst synthesis class code #c, namely, 0.8.

[0458] Determined in step S105 is the rate of change (ratio) of eachnegative tap coefficient of the set of tap coefficients of the secondsynthesis class code #c1 , namely, the second tap coefficient −0.02, andthe fourth tap coefficient −0.02, with respect to the corresponding tapcoefficient of the first synthesis class code #c.

[0459] Since the second and fourth tap coefficients of the firstsynthesis class code #c are respectively −0.1 and −0.1, the rates ofchange of the second and fourth tap coefficients are respectively0.02/0.1 and 0.02/0.1.

[0460] The mean rate of change v is thus 0.2 (step S105).

[0461] The mean rate of change v with a negative sign (−) sign attachedthereto, namely, −0.2 is the tap generation information (for the secondsynthesis class code #c1 ) for generating, from the set of tapcoefficients of the first synthesis class code #c, the set of tapcoefficients of the second synthesis class code #c1 corresponding to theset of tap coefficients of the first synthesis class code #c (stepS106).

[0462] The process of the coefficient generator 136 illustrated in FIG.32 is discussed below with reference to a flow diagram illustrated inFIG. 42. In this process, the set of tap coefficients of the secondsynthesis class code #c0 based on the first and second classclassification is generated from the set of tap coefficients of thefirst synthesis class code #c based on the first class classification inaccordance with the tap generation information generated in the processillustrated in FIG. 41.

[0463] The coefficient generator 136 determines in step S111 the sign ofthe tap generation information about the second synthesis class code #c0. If it is determined that the sign is positive (+), the algorithmproceeds to step S112. If it is determined that the sign is negative(−), the algorithm proceeds to step S114.

[0464] In step S112, the coefficient generator 136 multiplies thepositive tap coefficients, out of the tap coefficients of the firstsynthesis class code #c correspondingly associated with the secondsynthesis class code #c0 , by the mean rate of change v of the tapgeneration information, and regards the resulting set of tapcoefficients as the set of tap coefficients of the second synthesisclass code #c0 . The algorithm proceeds to step S113.

[0465] In step S113, the coefficient generator 136 performs anormalization process on the set of tap coefficients of the secondsynthesis class code #c0 determined in step S112 for gain adjustment,and ends the process. Specifically, in step S113, the coefficientgenerator 136 adjusts the negative tap coefficients out of the set oftap coefficients of the second synthesis class code #c0 determined instep S112 so that the sum of the tap coefficients becomes 1.

[0466] In step S114, the coefficient generator 136 multiplies thenegative tap coefficients, out of the set of tap coefficients of thefirst synthesis class code #c correspondingly associated with the secondsynthesis class code #c0 , by the mean rate of change v of the tapgeneration information, and regards the resulting products as the set oftap coefficients of the second synthesis class code #c0 . The algorithmproceeds to step S115.

[0467] In step S115, the coefficient generator 136 performs anormalization process on the set of tap coefficients of the secondsynthesis class code #c0 determined in step S114 for gain adjustment,and ends the process. Specifically, in step S115, the coefficientgenerator 136 adjusts the positive tap coefficients out of the set oftap coefficients of the second synthesis class code #c0 determined instep S112 so that the sum of the tap coefficients becomes 1.

[0468] In accordance with the process illustrated in FIG. 42, the set oftap coefficients of the first synthesis class code #c is (0.2, −0.1,0.8, −0.1, 0.2) as illustrated in FIG. 40, and the tap generationinformation of the second synthesis class code #c is +1.5 as illustratedin FIG. 41. The sign of the tap generation information is positive (+),and the positive tap coefficients out of the set of tap coefficients ofthe first synthesis class code #c (0.2, −0.1, 0.8, −0.1, 0.2) aremultiplied by 1.5, and the resulting products (0.3, −0.1, 1.2, −0.1,0.3) are handled as the set of tap coefficients of the tap coefficientsof the second synthesis class code #c0 (step S112). The negative tapcoefficients out of the set of the tap coefficients of the secondsynthesis class code #c0 (0.3, −0.1, 1.2, −0.1, 0.3) is adjusted so thatthe sum of the tap coefficients becomes 1. Specifically, the sum of(0.3, −0.1, 1.2, −0.1, 0.3) is 1.6, and each of the second and fourthnegative tap coefficients is adjusted from −0.1 to −0.4 by the sameamount. The set of the tap coefficients of the second synthesis classcode #c0 thus becomes (0.3, −0.4, 1.2, −0.4, 0.3) which causes the sumto be 1.

[0469] The set of the tap coefficients of the second synthesis classcode #c0 (0.3, −0.4, 1.2, −0.4, 0.3) coincides with the original set oftap coefficients illustrated in FIG. 40. The set of the tap coefficientsof the second synthesis class code #c0 (0.3, −0.4, 1.2, −0.4, 0.3)correspondingly associated with the first synthesis class code #c isgenerated from the set of tap coefficients of the first synthesis classcode #c (0.2, −0.1, 0.8, −0.1, 0.2) and the tap generation informationof +1.5. Similarly, the set of tap coefficients of the second synthesisclass code #c1 is generated from the set of tap coefficients of thefirst synthesis class code #c correspondingly associated therewith andthe tap generation information −1.2 which is generated as illustrated inFIG. 41.

[0470] As described above, each of the video processor cards 13 ₃through 13 ₆ stores the tap coefficients generated from the tapgeneration information of the video processor card 13 _(i) which is oneorder higher, together with tap generation information for generatingnew tap coefficients.

[0471] In the embodiment illustrated in FIG. 41, a positive sign (+) ora negative sign (−) is simply attached to the mean rate of change,thereby becoming the tap generation information of the second synthesisclass code #c0 . Alternatively, the mean rate of change is corrected byat least one of the second synthesis class code #c0 , the firstsynthesis class code #c correspondingly associated therewith, and thefirst class code and the second class code used to generate the secondsynthesis class code #c0 , and a positive sign (+) or a negative sign(−) is attached to the corrected mean rate of change. The resulting meanrate of change is used as the tap generation information for the secondsynthesis class code #c0 .

[0472] In the embodiment illustrated in FIG. 41, if the center tapcoefficient of the second synthesis class code #c0 is larger than thecenter tap coefficient of the first synthesis class code #c, the meanrate of change v of the positive tap coefficients of the secondsynthesis class code #c0 is determined, and if the center tapcoefficient of the second synthesis class code #c0 is not larger thanthe center tap coefficient of the first synthesis class code #c, themean rate of change v of the negative tap coefficients of the secondsynthesis class code #c0 is determined. A positive sign or negative signis attached to the resulting mean rate of change to obtain the tapgeneration information (hereinafter referred to as first tap generationinformation). Conversely, if the center tap coefficient of the secondsynthesis class code #c0 is not larger than the center tap coefficientof the first synthesis class code #c, the mean rate of change v of thepositive tap coefficients of the second synthesis class code #c0 isdetermined, and if the center tap coefficient of the second synthesisclass code #c1 is larger than the center tap coefficient of the firstsynthesis class code #c, the mean rate of change v of the negative tapcoefficients of the second synthesis class code #c0 is determined. Apositive sign or negative sign is attached to the resulting mean rate ofchange to obtain the tap generation information (hereinafter referred toas second tap generation information).

[0473] When the set of tap coefficients of the second synthesis classcode #c0 is determined using the tap generation information of thesecond synthesis class code #c0 in the process illustrated in FIG. 42,the resulting tap coefficients may deviate from the original ones. Thefirst tap generation information or the second tap generationinformation, whichever provides a smaller amount of deviation (error)between the set of tap coefficients of the second synthesis class code#c0 determined therefrom and the original values, may be adopted asfinal tap generation information. Whether to adopt the first tapgeneration information or the second tap generation information may bedetermined at each of the second synthesis class code, the firstsynthesis class code correspondingly associated therewith, and the firstclass code and the second class code used to generate the secondsynthesis class code.

[0474] The tap generation information for generating, from the set oftap coefficients of the first synthesis class code #c, the set of tapcoefficients of the second synthesis class code #c0 correspondinglyassociated with the set of tap coefficients of the first synthesis classcode #c may be the one generated using the process illustrated in FIG.41. Alternatively, the learning device illustrated in FIG. 9 performs alearning process using the set of tap coefficients of the firstsynthesis class code #c as the learning data, and the set of tapcoefficients of the second synthesis class code #c1 as the trainingdata, and the resulting set of tap coefficients is used as the tapgeneration information.

[0475]FIG. 43 illustrates a third construction of the video processinginterface 40 illustrated in FIG. 3. Components identical to thosediscussed with reference to FIG. 32 are designated with the samereference numerals, and the discussion thereof is omitted as appropriatehere. The video processing interface 40 illustrated in FIG. 43 isidentical in construction to the one illustrated in FIG. 32, except thata shared memory space controller 141 is substituted for the coefficientmemory 134.

[0476] Like the shared memory space controller 100 illustrated in FIG.20, the shared memory space controller 141 reserves a virtual memoryspace, for storing a set of tap coefficients generated by thecoefficient generator 136, in the coefficient memory 94 of a videoprocessor card 13 loaded in the video processing interface 40 asillustrated in FIG. 44. The shared memory space controller 141 storesthe set of tap coefficients in the virtual memory space. Furthermore,the shared memory space controller 141 reads, from the virtual memoryspace, a set of tap coefficients of a class code output from the classclassifier 133, and feeds the set of tap coefficients to the predictor135.

[0477]FIG. 44 illustrates the construction of the video processor card13 compatible with the video processing interface 40 constructed asillustrated in FIG. 43. As shown, components identical to thosedescribed with reference to FIG. 33 are designated with the samereference numerals, and the discussion thereof is omitted here asappropriate. The video processor card 13 illustrated in FIG. 44 includesthe video processor card 13 illustrated in FIG. 33 with the coefficientmemory 94 illustrated in FIG. 20 added thereto.

[0478] As discussed with reference to FIG. 36, as the number of videoprocessor card 13 ₁ through 13 _(i) loaded in the video processinginterface 40 as effective cards increases, the number of classes of tapcoefficients generated by the coefficient generator 136 in the videoprocessing interface 40 increases, and the set of tap coefficientsbecomes larger in size.

[0479] When the video processor card 13 ₁ through 13 _(i) are loaded inthe video processing interface 40 in the embodiment illustrated in FIG.44, the memory capacity of the coefficient memory 94 of each of thevideo processor card 13 ₁ through 13 _(i) is designed to match the sizeof the set of tap coefficients generated by the coefficient generator136 of the video processing interface 40. Even if the storage content ofthe video processor card 13 ₁ through 13 _(i) is illegally read togenerate the set of tap coefficients, the set of tap coefficients cannotbe stored unless the video processor card 13 ₁ through 13 _(i) areloaded in the video processing interface 40. In this way, the userprevents the storage content of the video processor card 13 ₁ through 13_(i) from being illegally read, thereby precluding an unauthorized useof the generated set of tap coefficients.

[0480] When the shared memory space controller 141 illustrated in FIG.43 is viewed from the class classifier 133, coefficient generator 136,and predictor 135, the shared memory space controller 141 reserves avirtual memory space so that the real memory spaces of i units ofcoefficient memories 94 of respective video processor card 13 ₁ through13 _(i) as effective cards loaded in the video processing interface 40look like a single consecutive memory space.

[0481] In this case, the capacity of the coefficient memory 94 isdesigned so that the total sum of capacities of the coefficient memories94 of the video processor card 13 ₁ through 13 _(i) loaded in the videoprocessing interface 40 matches the size of the generated tapcoefficients. The capacity of the coefficient memory 94 may be designedso that the size of tap coefficients generated with the video processorcard 13 ₁ through 13 _(i) loaded in the video processing interface 40 islarger than the capacity of (i−1) coefficient memories 94 but equal toor smaller than the capacity of i coefficient memories 94.

[0482] In the embodiments illustrated in FIGS. 33 and 44, the classclassifier 93 is mounted on the video processor card 13 _(i).Alternatively, the class classifier 93 may be mounted on the videoprocessing interface 40 illustrated in FIGS. 32 and 44.

[0483] The above series of process steps is performed using hardware.Alternatively, these process steps may be performed using softwareprograms. When the process steps are performed using a software program,the software program may be installed in a general-purpose computer.

[0484]FIG. 45 illustrates one embodiment of a computer in which theprogram for performing a series of process steps is installed.

[0485] The program may be stored beforehand in a hard disk 305 or a ROM303 as a storage medium built in the computer.

[0486] Alternatively, the program may be temporarily or permanentlystored (recorded) in a removable storage medium 311, such as a flexibledisk, CD-ROM (Compact Disk Read-Only Memory), MO (Magneto-optical) disk,DVD (Digital Versatile Disk), magnetic disk, or semiconductor memory.The removable storage medium 311 may be supplied in a so-called packagedsoftware.

[0487] The program may be installed in the computer using the removablestorage medium 311. Alternatively, the program may be radio transmittedto the computer from a down-load site via an artificial satellite fordigital broadcasting, or may be transferred to the computer in a wiredfashion using a network such as a LAN (Local Area Network) or theInternet. The computer receives the program at a communication unit 308,and installs the program in the built-in hard disk 305.

[0488] The computer contains a CPU (Central Processing Unit) 302. Aninput/output interface 310 is connected to the CPU 302 through a bus301. The CPU 302 carries out the program stored in the ROM (Read-OnlyMemory) 303 when the CPU 302 receives a command from a user through theinput/output interface 310 when the user operates an input unit 307 suchas a keyboard, mouse, or microphone. The CPU 302 carries out the programby loading on a RAM (Random Access Memory) 304, the program stored inthe hard disk 305, the program transmitted via a satellite or a network,received by the communication unit 308, and installed onto the hard disk305, or the program read from the removable storage medium 311 loadedinto a drive 309 and installed onto the hard disk 305. The CPU 302carries out the process in accordance with each of the above-referencedflow diagrams, or the process carried out by the arrangement illustratedin the above-referenced block diagrams. The CPU 302 outputs the resultsof the process from an output unit 306 such as a LCD (Liquid-CrystalDisplay) or a loudspeaker through the input/output interface 310, ortransmits the results of the process through the communication unit 308,or stores the results of the process onto the hard disk 305.

[0489] It is not a requirement that the process steps describing theprogram for causing the computer to carry out a variety of processes becarried out in a sequential order in time scale described in the flowdiagrams. The process steps may be performed in parallel or separately(for example, parallel processing or processing using an object).

[0490] The program may be executed by a single computer, or by aplurality of computers in distributed processing. The program may betransferred to and executed by a computer at a remote place.

[0491] In the above-referenced embodiments, the present invention isapplied to the image quality improvements of video data. The presentinvention may be applied to sound quality improvements of sound (audio)data.

[0492] In the above embodiments, the present invention is applied toanalog television receivers. Alternatively, the present invention may beapplied to digital television receivers, or VCRs (Video CassetteRecorders) for processing video data and audio data.

[0493] In the above-referenced embodiments, the tap generationinformation for generating the tap coefficients for improving spatialresolution is stored in the video processor card 13. The tap generationinformation stored in the video processor card 13 is not limited to thatinformation. The video processor card 13 may store a tap coefficient forimproving time resolution, a tap coefficient for improving tonalgradations of pixels (the number of bits of a pixel value), a tapcoefficient for edge enhancement, a tap coefficient for converting thenumber of pixels forming an image or for converting the size of animage, or a tap coefficient for another type of image improvements, forexample.

[0494] The tap generation information stored in the video processorcards 13 ₁ through 13 _(i) may perform the same type of image qualityimprovements at different levels of improvements, or may performdifferent types of image quality improvements.

[0495] In the above-referenced embodiments, the video processor card 13stores beforehand the tap generation information. The video processorcard 13 _(i) may be sold with no particular tap generation informationstored therein. In this case, tap generation information may bedownloaded from a tap generation information server, and is then storedin the video processor card 13 _(i). When the video processor card 13_(i) is loaded in the video processing interface 40, the controller 37in FIG. 3 controls the communication interface 38, thereby accessing thetap generation information server. After the controller 37 and the tapgeneration information server authenticate each other, the tapgeneration information is downloaded. The bill for the downloaded tapgeneration information is debited from the bank account of the user, ormay be included in the retail price of the video processor card 13 _(i).

[0496] As lower order video processor card 13 is loaded, the functionand/or performance of the main unit 1 becomes sophisticated. The retailprice of the video processor card 13 _(i) or the charge for downloadingthe tap generation information may increase as the video processor card13 _(i) rises in order.

[0497] In the above-referenced embodiments, the television receiver hasthe highest order video processor card 13 ₁ already mounted therewithin.The highest order video processor card 13 ₁ may be an optional item. Auser, who has purchased the television receiver with the highest ordervideo processor card 13 ₁ mounted therewithin, may enjoy a more pricediscount when he or she purchases a lower order video processor card 13_(i) than a user who purchases a television receiver without the highestorder video processor card 13 ₁.

[0498] In principle, the main unit 1 of the television receiver shown inFIG. 3 is produced by adding the video processing interface 40 to atypically available analog television receiver, and modifying a programwhich is executed by the controller 37. The main unit 1 of thetelevision receiver shown in FIG. 3 is manufactured using a typicalanalog television receiver with relative ease. The cost performance ofthe video processor card 13 is high considering the above-referencedfunctions.

[0499] In accordance with the present invention, a television receiveris provided with sophisticated functions in an add-on fashion.

[0500] the data processing apparatus comprises:

[0501] loading and unloading means on which each of the first throughN-th storage devices is mounted;

[0502] tap coefficient generator means which generates the tapcoefficient from the tap generation information in the first throughN′-th storage devices (N′≦N) mounted on the loading and unloading means;

[0503] predictive tap extractor means which extracts the first datahaving a predictive tap which is used to predict target data which is ofinterest in the second data;

[0504] class tap extractor means which extracts the first data having aclass tap which is used to classify the target data into one of aplurality of classes;

[0505] class classifier means which classifies the target data based onthe class tap; and

[0506] predictor means which predicts the target data based on the tapcoefficient and the predictive tap of the class of the target data.

What is claimed is:
 1. A storage device detachably loaded in a dataprocessing apparatus, comprising: tap generation information storagemeans which stores tap generation information for generating a tapcoefficient for each predetermined class for a data conversion processof converting first data into second data higher in quality level thanthe first data; tap coefficient generator means which generates the tapcoefficient from the tap generation information under the control of thedata processing apparatus; tap extractor means which extracts, from thefirst data supplied from the data processing apparatus, a predictive tapwhich is used to predict target data which is of interest in the seconddata; class tap extractor means which extracts, from the first datasupplied from the data processing apparatus, a class tap which is usedto classify the target data into one of a plurality of classes; classclassifier means which classifies the target data according to the classtap; and predictor means which predicts the target data from the tapcoefficient and the predictive tap of the class of the target data, andsupplies the data processing apparatus with the target data.
 2. Astorage device according to claim 1, wherein the tap generationinformation storage means stores, as the tap generation information,coefficient seed data that serves as a seed of the tap coefficient.
 3. Astorage device according to claim 2, wherein another storage device isdetachably loaded to the data processing apparatus, and wherein the tapgeneration information storage means stores, as the tap generationinformation, information that generates the coefficient seed data of thestorage device by using another coefficient seed data stored in theother storage device.
 4. A storage device according to claim 3, whereinthe tap generation information storage means stores, as the tapgeneration information, a difference between the coefficient seed dataof the storage device and the other coefficient seed data in the otherstorage device.
 5. A storage device according to claim 1, whereinanother storage device is detachably loaded in the data processingapparatus, and wherein the tap coefficient generator means generates thetap coefficient based on the tap generation information stored in thetap generation information storage means and the other tap generationinformation stored in the other storage device and supplied from thedata processing apparatus.
 6. A storage device according to claim 1,further comprising: tap coefficient storage means which store the tapcoefficient; and tap coefficient acquisition means which acquires thetap coefficient of the class of the target data.
 7. A storage deviceaccording to claim 6, wherein another storage device is detachablyloaded in the data processing apparatus, and wherein the tap coefficientgenerator means generates the tap coefficient having a size larger thanthe storage capacity of the tap coefficient storage means, the tapcoefficient generated by the tap coefficient generator means is storedin the tap coefficient storage means and the other storage device, andthe tap coefficient acquisition means acquires the tap coefficient ofthe class of the target data from one of the tap coefficient storagemeans and the other storage device.
 8. A storage device according toclaim 1, wherein the tap coefficient generator means generates the tapcoefficient by calculating a predetermined calculation equation definedby the tap generation information.
 9. A storage device according toclaim 8, wherein another storage device is detachably loaded in the dataprocessing apparatus, and wherein the tap coefficient generator meansgenerates the tap coefficient by calculating a predetermined calculationequation having the number of terms more than the calculation equationused in the other storage device.
 10. A storage device according toclaim 1, wherein another storage device is detachably loaded in the dataprocessing apparatus, and wherein the tap coefficient generator meansgenerates the tap coefficient having the number of classes more than thenumber of classes of the tap coefficient generated in the other storagedevice.
 11. A storage device according to claim 1, wherein anotherstorage device is detachably loaded in the data processing apparatus,and wherein the tap generation information storage means stores the tapgeneration information for generating a tap coefficient for the dataconversion process that performs quality improvements different fromquality improvements resulting from the tap coefficient generated in theother storage device.
 12. A data processing method of a storage devicedetachably loaded in a data processing apparatus, the storage devicehaving tap generation information storage means for storing tapgeneration information for generating a tap coefficient for eachpredetermined class for a data conversion process of converting firstdata into second data higher in quality level than the first data, thedata processing method comprising: a tap coefficient generation step ofgenerating the tap coefficient from the tap generation information underthe control of the data processing apparatus; a predictive tapextraction step of extracting, from the first data supplied from thedata processing apparatus, a predictive tap for use in the prediction oftarget data which is of interest in the second data; a class tapextraction step of extracting, from the first data supplied from thedata processing apparatus, a class tap for use in the classclassification that classifies the target data into one of a pluralityof classes; a class classification step of classifying the target datainto classes based on the class tap; and a prediction step of predictingthe target data from the tap coefficient and the predictive tap of theclass of the target data, and supplying the target data to the dataprocessing apparatus.
 13. A computer program of a data processing methodof a storage device detachably loaded in a data processing apparatus,the storage device having tap generation information storage means forstoring tap generation information for generating a tap coefficient foreach predetermined class for a data conversion process of convertingfirst data into second data higher in quality level than the first data,the computer program comprising: a tap coefficient generation step ofgenerating the tap coefficient from the tap generation information underthe control of the data processing apparatus; a predictive tapextraction step of extracting, from the first data supplied from thedata processing apparatus, a predictive tap for use in the prediction oftarget data which is of interest in the second data; a class tapextraction step of extracting, from the first data supplied from thedata processing apparatus, a class tap for use in the classclassification that classifies the target data into one of a pluralityof classes; a class classification step of classifying the target datainto classes based on the class tap; and a prediction step of predictingthe target data from the tap coefficient and the predictive tap of theclass of the target data, and supplying the target data to the dataprocessing apparatus.
 14. A storage medium storing a computer program ofa data processing method of a storage device detachably loaded in a dataprocessing apparatus, the storage device having tap generationinformation storage means for storing tap generation information forgenerating a tap coefficient for each predetermined class for a dataconversion process of converting first data into second data higher inquality level than the first data, the computer program comprising: atap coefficient generation step of generating the tap coefficient fromthe tap generation information under the control of the data processingapparatus; a predictive tap extraction step of extracting, from thefirst data supplied from the data processing apparatus, a predictive tapfor use in the prediction of target data which is of interest in thesecond data; a class tap extraction step of extracting, from the firstdata supplied from the data processing apparatus, a class tap for use inthe class classification that classifies the target data into one of aplurality of classes; a class classification step of classifying thetarget data into classes based on the class tap; and a prediction stepof predicting the target data from the tap coefficient and thepredictive tap of the class of the target data, and supplying the targetdata to the data processing apparatus.
 15. A data processing apparatusthat allows first through N-th storage devices to be detachably loadedtherein, each of the storage devices storing tap generation informationfor generating a tap coefficient for each predetermined class for a dataconversion process of converting first data into second data higher inquality level than the first data, the data processing apparatuscomprising: loading and unloading means on which each of the firstthrough N-th storage devices is mounted; tap coefficient generationcontrol means which controls the generation of the tap coefficient fromthe tap generation information in the first through N′-th storagedevices (N′≦N) mounted on the loading and unloading means; input andoutput route setting means which sets an input and output route of datafor each of the first through N′-th storage devices; and data supplycontrol means which controls the supply of data from one storage deviceto another among the first through N′-th storage devices in accordancewith the input and output route set by the input and output routesetting means.
 16. A data processing apparatus according to claim 15,wherein the one storage device stores, as the tap generationinformation, coefficient seed data which functions as a seed for the tapcoefficient, wherein the other storage device stores, as the tapgeneration information, information for generating coefficient seed dataof the other storage device using the coefficient seed data stored inthe one storage device, wherein the tap coefficient generation controlmeans controls the other storage device to generate the coefficient seeddata of the other storage device using the coefficient seed data storedin the one storage device and the tap generation information stored inthe other storage device, and to generate the tap coefficient from thegenerated coefficient seed data.
 17. A data processing apparatusaccording to claim 16, wherein the other storage device stores, as thetap generation information, a difference between the coefficient seeddata of the other storage device and the coefficient seed data of theone storage device, and wherein the tap coefficient generation controlmeans controls the other storage device to generate the coefficient seeddata of the other storage device by summing the coefficient seed datastored in the one storage device and the tap generation informationstored in the other storage device, and to generate the tap coefficientfrom the generated coefficient seed data.
 18. A data processingapparatus according to claim 15, wherein the tap coefficient generationcontrol means controls the other storage device to generate the tapcoefficient based on the tap generation information stored in the onestorage device and the tap generation information stored in the otherstorage device.
 19. A data processing method of a data processingapparatus that allows first through N-th storage devices to bedetachably loaded therein, each of the storage devices storing tapgeneration information for generating a tap coefficient for eachpredetermined class for a data conversion process of converting firstdata into second data higher in quality level than the first data, thedata processing apparatus comprising loading and unloading means onwhich each of the first through N-th storage devices is mounted, thedata processing method comprising: a tap coefficient generation controlstep of controlling the generation of the tap coefficient from the tapgeneration information in the first through N′-th storage devices (N′≦N)mounted on the loading and unloading means; an input and output routesetting step of setting an input and output route of data for each ofthe first through N′-th storage devices; and a data supply control stepof controlling the supply of data from one storage device to anotheramong the first through N′-th storage devices in accordance with theinput and output route set in the input and output route setting step.20. A computer program for a data processing method of a data processingapparatus that allows first through N-th storage devices to bedetachably loaded therein, each of the storage devices storing tapgeneration information for generating a tap coefficient for eachpredetermined class for a data conversion process of converting firstdata into second data higher in quality level than the first data, thedata processing apparatus comprising loading and unloading means onwhich each of the first through N-th storage devices is mounted, thecomputer program comprising: a tap coefficient generation control stepof controlling the generation of the tap coefficient from the tapgeneration information in the first through N′-th storage devices (N′≦N)mounted on the loading and unloading means; an input and output routesetting step of setting an input and output route of data for each ofthe first through N′-th storage devices; and a data supply control stepof controlling the supply of data from one storage device to anotheramong the first through N′-th storage devices in accordance with theinput and output route set in the input and output route setting step.21. A storage medium storing a computer program for data processingmethod of a data processing apparatus that allows first through N-thstorage devices to be detachably loaded therein, each of the storagedevices storing tap generation information for generating a tapcoefficient for each predetermined class for a data conversion processof converting first data into second data higher in quality level thanthe first data, the data processing apparatus comprising loading andunloading means on which each of the first through N-th storage devicesis mounted, the computer program comprising: a tap coefficientgeneration control step of controlling the generation of the tapcoefficient from the tap generation information in the first throughN′-th storage devices (N′≦N) mounted on the loading and unloading means;an input and output route setting step of setting an input and outputroute of data for each of the first through N′-th storage devices; and adata supply control step of controlling the supply of data from onestorage device to another among the first through N′-th storage devicesin accordance with the input and output route set in the input andoutput route setting step.
 22. A data processing system comprising:first through N-th storage devices which store tap generationinformation for generating a tap coefficient for each predeterminedclass for a data conversion process of converting first data into seconddata higher in quality level than the first data, and a data processingapparatus that allows the first through N-th storage devices to bedetachably loaded therein; wherein each of the first through N-thstorage device comprises: tap generation information storage means whichstores the tap generation information for generating the tapcoefficient, tap coefficient generator means which generates the tapcoefficient from the tap generation information under the control of thedata processing apparatus, tap extractor means which extracts, from thefirst data supplied from the data processing apparatus, a predictive tapwhich is used to predict target data which is of interest in the seconddata, class tap extractor means which extracts, from the first datasupplied from the data processing apparatus, a class tap which is usedto classify the target data into one of a plurality of classes, classclassifier means which classifies the target data according to the classtap, and predictor means which predicts the target data from the tapcoefficient of and the predictive tap of the class of the target data,and supplies the data processing apparatus with the target data; andwherein the data processing apparatus comprises: loading and unloadingmeans on which each of the first through N-th storage devices ismounted, tap coefficient generation control means which controls thegeneration of the tap coefficient from the tap generation information inthe first through N′-th storage devices (N′≦N) mounted on the loadingand unloading means, input and output route setting means which sets aninput and output route of data for each of the first through N′-thstorage devices, and data supply control means which controls the supplyof data from one storage device to another among the first through N′-thstorage devices in accordance with the input and output route set by theinput and output route setting means.
 23. A storage device detachablyloaded in a data processing apparatus, comprising: tap generationinformation storage means which stores tap generation information forgenerating a tap coefficient for each predetermined class for a dataconversion process of converting first data into second data higher inquality level than the first data, the tap coefficient being generatedfrom the tap generation information stored in the storage device and thetap generation information stored in another storage device; and tapgeneration information supply means which supplies the data processingapparatus with the tap generation information.
 24. A storage deviceaccording to claim 23, wherein the tap generation information storagemeans stores, as the tap generation information, information forgenerating coefficient seed data serving as a seed of the tapcoefficient from the tap generation information stored in the otherstorage device.
 25. A storage device according to claim 24, wherein thetap generation information storage means stores, as the tap generationinformation, a difference between the coefficient seed data of thestorage device and the other coefficient seed data stored in the otherstorage device.
 26. A storage device according to claim 23, wherein thetap coefficient generated from the tap generation information stored inthe tap generation information storage means is larger in size than thetap coefficient generated from the tap generation information stored inthe other storage device.
 27. A storage device according to claim 23,wherein the tap coefficient is determined by performing calculationaccording to a predetermined calculation equation that is defined by thetap generation information stored in the tap generation informationstorage means.
 28. A storage device according to claim 27, wherein thepredetermined calculation equation defined by the tap generationinformation stored in the tap generation information storage means hasmore terms than a predetermined calculation equation defined by the tapgeneration information stored in the other storage device.
 29. A storagedevice according to claim 23, wherein the tap coefficient generated fromthe tap generation information stored in the tap generation informationstorage means has more classes than the tap coefficient generated fromthe tap generation information stored in the other storage device.
 30. Astorage device according to claim 23, wherein the tap coefficientgenerated from the tap generation information stored in the tapgeneration information storage means is for use in the data conversionprocess for quality improvements different from quality improvementsprovided by the tap coefficient generated from the tap generationinformation stored in the other storage device.
 31. A storage deviceaccording to claim 23, further comprising tap coefficient storage meanswhich stores a tap coefficient generated in the data processingapparatus and supplies the data processing apparatus with a tapcoefficient of a class corresponding to a request from the dataprocessing apparatus in response to the request when the data processingapparatus generates the tap coefficient from the tap generationinformation.
 32. A storage device according to claim 23, furthercomprising class classifier means which classifies target data based ona class tap which is first data to be used to classify the target data,which is of interest in the second data, in response to a request fromthe data processing apparatus, and supplies the data processingapparatus with information representing the class of the target data.33. A data processing apparatus that allows first through N-th storagedevices to be detachably loaded therein, each of the storage devicesstoring tap generation information for generating a tap coefficient foreach predetermined class for a data conversion process of convertingfirst data into second data higher in quality level than the first data,the data processing apparatus comprising: loading and unloading means onwhich each of the first through N-th storage devices is mounted; tapcoefficient generator means which generates the tap coefficient from thetap generation information in the first through N′-th storage devices(N′≦N) mounted on the loading and unloading means; predictive tapextractor means which extracts the first data having a predictive tapwhich is used to predict target data which is of interest in the seconddata; class tap extractor means which extracts the first data having aclass tap which is used to classify the target data into one of aplurality of classes; class classifier means which classifies the targetdata based on the class tap; and predictor means which predicts thetarget data based on the tap coefficient and the predictive tap of theclass of the target data.
 34. A data processing apparatus according toclaim 33, wherein the second data which is obtained using the tapcoefficient generated from the tap generation information in the firstthrough N′-th storage devices is higher in quality level than the seconddata which is obtained using the tap coefficient generated from the tapgeneration information in the first through (N′−1)-th storage devices.35. A data processing apparatus according to claim 33, wherein the tapcoefficient generated from the tap generation information in the firstthrough N′-th storage devices is different from the tap coefficientgenerated from the tap generation information in the first through(N′−1)-th storage devices.
 36. A data processing apparatus according toclaim 33, wherein the tap coefficient generator means generates the tapcoefficient by performing calculation in accordance with a predeterminedcalculation equation defined by the tap generation information stored inthe first through N′-th storage devices.
 37. A data processing apparatusaccording to claim 36, wherein the calculation equation defined by thetap generation information in the first through N′-th storage deviceshas more terms than the calculation equation defined by the tapgeneration information in the first through (N′−1)-th storage devices.38. A data processing apparatus according to claim 33, wherein when eachof the first through N-th storage devices comprises tap coefficientstorage means for storing the tap coefficient generated by the tapcoefficient generator means, the size of the tap coefficient generatedfrom the tap generation information in the first through N′-th storagedevices is larger than the overall storage capacity of the tapcoefficient storage means of the first through (N′−1)-th storagedevices, and is equal to or smaller than the overall storage capacity ofthe tap coefficient storage means in the first through N′-th storagedevices.
 39. A data processing apparatus according to claim 38, whereinwhen each of the first through N-th storage devices comprises tapcoefficient storage means for storing the tap coefficient generated bythe tap coefficient generator means, the tap coefficient generated fromthe tap generation information in each of the first through N′-thstorage devices is stored straddling the tap coefficient storage meansin the first through N′-th storage devices.
 40. A data processingapparatus according to claim 33, wherein the tap coefficient generatedfrom the tap generation information stored in the first through N′-thstorage devices has more classes than the tap coefficient generated fromthe tap generation information in the first through (N′−1)-th storagedevices.
 41. A data processing apparatus according to claim 33, whereinthe tap coefficient generator means generates a set of tap coefficientsfrom the tap generation information in one of the first through N′-thstorage devices.
 42. A data processing apparatus according to claim 41,wherein the one set of tap coefficients generated from the tapgeneration information in the first storage device performs qualityimprovements of N types when the first storage device only is mounted onthe loading and unloading means, and wherein when the first throughN′-th storage devices, out of the first through N-th storage devices,are mounted on the loading and unloading means, N sets of the tapcoefficients respectively generated from the tap generation informationin the first through N-th storage devices respectively correspond to thequality improvements of N types.
 43. A data processing apparatusaccording to claim 41, wherein the tap coefficients of N′ typesrespectively generated from the tap generation information in the firstthrough N′-th storage devices are used to perform the data conversionprocess on the first data by N times.
 44. A data processing apparatusaccording to claim 33, wherein the tap coefficient generator meansgenerates the tap coefficient from coefficient seed data serving as aseed of the tap coefficient, the coefficient seed data being the tapgeneration information in the first through N′-th storage devices.
 45. Adata processing apparatus according to claim 44, wherein the N′-thstorage device stores, as the tap generation information, informationfor generating N′-th coefficient seed data using (N′−1)-th coefficientseed data determined from the tap generation information in the firstthrough (N′−1)-th storage devices, and wherein the tap coefficientgenerator means generates the N′-th coefficient seed data from the tapgeneration information in the first through N′-th storage devices, andfurther generates the tap coefficient from the N′-th coefficient seeddata.
 46. A data processing apparatus according to claim 45, wherein theN′-th storage device stores, as the tap generation information, adifference between the N′-th coefficient seed data and the (N′−1)-thcoefficient seed data, and the tap coefficient generator means generatesthe N′-th coefficient seed data by summing the (N′−1)-th coefficientseed data determined from the tap generation information in the firstthrough (N′−1)-th storage devices and the tap generation information inthe N′-th storage device.
 47. A data processing apparatus according toclaim 33, further comprising class classification requesting means whichrequests the first through N′-th storage devices to classify the targetdata, wherein the class classifier means determines a final class of thetarget data by synthesizing information representing the class of thetarget data respectively supplied from each of the first through N′-thstorage devices in response to the class classification request.
 48. Adata processing method of a data processing apparatus that allows firstthrough N-th storage devices to be detachably loaded therein, each ofthe storage devices storing tap generation information for generating atap coefficient for each predetermined class for a data conversionprocess of converting first data into second data higher in qualitylevel than the first data, the data processing method comprising: a tapcoefficient generation step of generating the tap coefficient from thetap generation information in the first through N′-th storage devices(N′≦N) loaded in the data processing apparatus; a predictive tapextracting step of extracting the first data having a predictive tapwhich is used to predict target data which is of interest in the seconddata; a class tap extracting step of extracting the first data having aclass tap which is used to classify the target data into one of aplurality of classes; a class classifying step of classifying the targetdata based on the class tap; and a predicting step of predicting thetarget data based on the tap coefficient and the predictive tap of theclass of the target data.
 49. A computer program of a data processingmethod of a data processing apparatus that allows first through N-thstorage devices to be detachably loaded therein, each of the storagedevices storing tap generation information for generating a tapcoefficient for each predetermined class for a data conversion processof converting first data into second data higher in quality level thanthe first data, the computer program comprising: a tap coefficientgeneration step of generating the tap coefficient from the tapgeneration information in the first through N′-th storage devices (N′≦N)loaded in the data processing apparatus; a predictive tap extractingstep of extracting the first data having a predictive tap which is usedto predict target data which is of interest in the second data; a classtap extracting step of extracting the first data having a class tapwhich is used to classify the target data into one of a plurality ofclasses; a class classifying step of classifying the target data basedon the class tap; and a predicting step of predicting the target databased on the tap coefficient and the predictive tap of the class of thetarget data.
 50. A storage medium storing a computer program for a dataprocessing method of a data processing apparatus that allows firstthrough N-th storage devices to be detachably loaded therein, each ofthe storage devices storing tap generation information for generating atap coefficient for each predetermined class for a data conversionprocess of converting first data into second data higher in qualitylevel than the first data, the computer program comprising: a tapcoefficient generation step of generating the tap coefficient from thetap generation information in the first through N′-th storage devices(N′≦N) loaded in the data processing apparatus; a predictive tapextracting step of extracting the first data having a predictive tapwhich is used to predict target data which is of interest in the seconddata; a class tap extracting step of extracting the first data having aclass tap which is used to classify the target data into one of aplurality of classes; a class classifying step of classifying the targetdata based on the class tap; and a predicting step of predicting thetarget data based on the tap coefficient and the predictive tap of theclass of the target data.
 51. A data processing system comprising: firstthrough N-th storage devices storing tap generation information forgenerating a tap coefficient for each predetermined class for a dataconversion process of converting first data into second data higher inquality level than the first data, and a data processing apparatus onwhich the first through N-th storage devices are detachably loaded;wherein each of the first through N-th storage devices comprises tapgeneration information storage means which stores tap generationinformation for generating the tap coefficient, the tap coefficientbeing generated from the tap generation information and tap generationinformation stored in another storage device; and tap generationinformation supply means which supplies the data processing apparatuswith the tap generation information; and